• Home

Items filtered by date: December 2015

Friday, 06 February 2015 21:07

Redmine Security and SSL

Security and SSL

If you are using this on a real website you might want to use SSL and make your installation secure. There are a few things you can do. For most people running on their local machine that do not want it on the internet do not require any of these settings, but it will not harm if they did. Following the instructions above you should already be using ProxyRequests Off

<Directory> / <Location>

  • location acts upon the url only, directory works on the physical file path, so they basically do the same thing via the 2 declarative paths.
  • however, directory is a little more secure because there can be multip-le urls poitning to the same file whereas when using directory there is only 1 file path so directory is better for security permissions etc ..
  • directory and location directives can be used within a virtual host and will apply to that virtualhost only, if not it will be applied to the main/default website/name/localhost etc..
  • using <directory> is better than <location>

SSL on non default port

The optional protocol argument is not required for most configurations. If not specified, https is the default for port 443 and http the default for all other ports. The protocol is used to determine which module should handle a request, and to apply protocol specific optimizations with the AcceptFilter directive.

You only need to set the protocol if you are running on non-standard ports. For example, running an https site on port 8443:

Listen 192.170.2.1:8443 https

Hooks

Secure Proxy

apache httpd.conf

<Proxy *>
  Order Deny,Allow
  Deny from all
  Allow from 192.168.0
</Proxy>
Published in Redmine
Read more...
Tuesday, 03 February 2015 19:35

Running Redmine Through Apache on Xampp

Some Notes Before We Start

When you have a webserver running you probably would like redmine through that server aswell to keep things easier, reducing port forwarding requirements and the number of addresses you hand out. You could just hand out the new address which usually looks like http://localhost:3000/ if you want, but why not do a proper integration.

These methods will allow you to run redmine from your normal webserver or localhost, http://localhost/ with no port number on the end (this assumes that your webserver does not have a port number, using port 80).

Tthere are 2 parts of this process to consider:

  1. Changing the folder from which redmine is server ie from http://localhost:3000/ to http://localhost:3000/redmine/
  2. Proxying / Redirecting redmine from the server where it is (ie http://localhost:3000/) to youer apache server http://localhost/

With the following instructions i will assume the following

If you just want the code, click here, this will take you to the bottom where you can just get the code for a quick start without any of the history.

Files

This section is not required reading but might help if you have issues or question later on.

During my research i cam across references to Redmine files where you could put code. I am going to list them here and what i think they do.

  • {redmine}/config.ru - this is the intialization configuration file, i think
  • {redmine}/config/enviroment.rb - This file controls additonal settings that will get applied to both development and production enviroments
  • {redmine}/config/additional_enviroment.rb - This is only present in later version of Ruby/Rails/Redmine and probably needs creating. . This file controls additonal settings that will get applied to both development and production enviroments similiar enviroment.rb does. Not all commands in this file work in this file that would in enviroment.rb .
  • {redmine}/config/application.rb - dont bother altering code in this
  • {redmine}/config/development.rb - this files allows you to add additional commands/instructions to just the development enviroment
  • {redmine}/config/production.rb - this files allows you to add additional commands/instructions to just the production enviroment

Move Redmine to a Sub-Folder

Method 1 - Using WebBrick

This techniques is the best an i have taken most of the information from Defect #12102: Installed Redmine in a sub-URI: Links are generated not correctly - Redmine

This method also utilises the inbuilt WebBrick Webserver, This is an excellent simple solution but if you are going to use this for a lot of users you should perhaps think about using Thin Server, this will require slightly different instructions and some more work. WebBrick is classed as a reference server and if you are going to server more than 20 people at a time there will be large performance issues. For the most of us WebBrick will be fine.

config.ru

# This file is used by Rack-based servers to start the application.

require ::File.expand_path('../config/environment',  __FILE__)
map '/redmine' do
  run RedmineApp::Application
end

Replace the contents of {redmine}/config.ru with this code. This also assumes you have not already made any alterations to this file.

This code does 2 things

  1. changes all links in the pages to /redmine/
  2. changes the actual served root. Files in the /public/ directory are now served as http://3000/redmine/ instead of http://localhost:3000/

The root folder effectively no longer exists, you get page not found supplied by the webserver if you browse to http://localhost:3000/ and this is normal.

Other Static Asset Commands

You can also change the dynamic urls by using the 2 following commands:

  • RedmineApp::Application.routes.default_scope = { :path => '/redmine', :shallow_path => '/redmine'
  • RedmineApp::Application.routes.default_scope = '/redmine'

which cause the following issues:

  • the directory of the dynamic links changes but the root of the webserver stays the same. This means that the static files stay in the root directory (http://localhost/) while all of the dynamic links are in http://localhost:3000/redmine/
  • it is very difficult to proxy these 2 folders because this will involve complex rewrite rules

enviroment.rb 

ActionController::Base.relative_url_root = '/redmine'

Add this to the end of {redmine}config/enviroment.rb. I also think you can put this in either development.rb or production.rb, this change will then only occur in that particular enviroment.

This code alters the path of the links pointing to the static files such as javascripts and CSS. Currently all paths for these still point to the root http://localhost:3000/ but they actual ar now present at http://localhost:3000/redmine/ and even though we have added the code in config.ru these paths are not altered so we need this code to correct that.

NB:

  • RedmineApp::Application.routes.default_scope = '/redmine' works but is not the prefered method. This must also be put before RedmineApp::Application.initialize!
  • ActionController::AbstractRequest.relative_url_root = "/redmine" - Does not work, it was for older versions of Redmine
  • config.action_controller.relative_url_root = '/redmine' - not sure about this

You now have moved your Redmine installation to a subfolder

Method 2 - Thin Server + Passenger Server

this will be for my Thin server notes when i do them, there is also the Passenger Webserver to consider. Each has there own method. Basically youo can run these servers with switches to perform the directory change, You can then proxy this over to your main webserver as in the instructions below. Check my General Links section for resources.

Proxying Redmine to Apache / Xampp

Redmine in Root Folder

This is the easiest of the methods to implement. You can simply proxy Remine to your apache server using the following code, this assumes redmine is at http://localhost:3000/

httpd.conf

LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so

Make sure these modules are enabled in httpd.conf, they are required.

ProxyRequests Off
ProxyPreserveHost On
ProxyPass / http://localhost:3000/
ProxyPassReverse / http://localhost:3000/

Add these lines to the end your apache httpd.conf , if you do this you will not be able to run any other stuff from your apache webserver because all requests will get forwarded to the Redmine server.

Or Alter the Redmine Server port directely

Simply run the Redmine server on port 80 by using the following switch which will give the same effect

-p 80

Redmine in Subfolder

Method 1 - Basic (also blunt and site wide)

These instructions will assume the subfolder on apache you want is http://localhost/redmine/

.htaccess file

# Redirects /redmine to /redmine/
RedirectMatch ^/redmine$ /redmine/
RedirectMatch ^/$ /redmine/

add this code to ..../htdocs/.htaccess in your apache server.

You can redirect the non-slashed version, but this is to make sure we have a correct URL. A directory is denoted by having a slash at the end of the URL (ie /redmine/ ) wheresas if you dont, it denotes a page (ie /redmine ).

What we are directing to is more akin to a directory/default index file like a normal webserver, http://www.example.com/ You dont often see http://www.example.com/index.html , but they are the same page. Notice the slash at the end, this is intentional.

httpd.conf

LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so

Make sure these modules are enabled in httpd.conf, they are required.

ProxyRequests Off
ProxyPreserveHost On
ProxyPass /redmine/ http://localhost:3000/redmine/
ProxyPassReverse /redmine/ http://localhost:3000/redmine/

This is code that actualy passes the information/request between Redmine Server and your Apache server.

Can I make this Better

The commands above will work site wide so are a bit blunt, the following methods will allow you to apply these rules to a specific locations.

Method 2 - <Location>

<Location> works on the URLs presented to the server

httpd.conf

LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so

Make sure these modules are enabled in httpd.conf, they are required.

ProxyRequests Off	
<Location /redmine/>
	RedirectMatch ^/redmine$ /redmine/
	RedirectMatch ^/$ /redmine/
	ProxyPreserveHost On
	ProxyPass http://localhost:3000/redmine/
	ProxyPassReverse http://localhost:3000/redmine/
</Location>
  • ProxyRequests Off will not work in <Location>
  • Notice that the proxy statements do not have a target in them, only the source
  • The RedirectMatch commands will work outside of <Location>
  • rules in <location> can be moved easier that <directory> because there are not absolut addresses

With this method you have to split the commands, ProxyRequests will not work in location so needs to be put in the normal flow of the httpd.conf (i.e. not in <location>)

Method 3 - <Directory>

Directory works on the physical path of the accessed files and is better to be used for security because several URLs can point to a single directory or file whereas the <Directory> command is very specific.

httpd.conf

LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so

Make sure these modules are enabled in httpd.conf, they are required.

ProxyRequests Off
<Directory "d:\Documents\Developer\xampp\htdocs\redmine\">
	RedirectMatch ^/redmine$ /redmine/
	RedirectMatch ^/$ /redmine/
	ProxyPreserveHost On
	ProxyPass http://localhost:3000/redmine/
	ProxyPassReverse http://localhost:3000/redmine/
</Directory>
  • ProxyRequests Off will not work in <Location>
  • Notice that the proxy statements do not have a target in them, only the source
  • <Directory "D:/Documents/Developer/xampp/htdocs/redmine/"> is also valid
  • this command acts on the Virtual Directory in the xampp public folder
  • This should work even though the directory does not exist
  • The RedirectMatch commands will work outside of <Directory>

Method 4 - VirtualHost

With Virtualhost you can run a another domain name on the 1 xampp/apache server. This is very useful. You can also just use it to apply rules to the server and keep the code neater, easier for copying and pasting.

httpd.conf

LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so

Make sure these modules are enabled in httpd.conf, they are required.

<VirtualHost *:80>

	## This is for setting a domain name
	ServerName rails.localhost.com
	DocumentRoot "D:\Documents\Developer\RubyRoot\redmine\public\"

	# Make sure there is a trailing slash in the root
	RedirectMatch ^/redmine$ /redmine/
	RedirectMatch ^/$ /redmine/
	
	# The proxying code
	ProxyRequests Off 
	ProxyPreserveHost On
	ProxyPass /redmine/ http://localhost:3000/redmine/
	ProxyPassReverse /redmine/ http://localhost:3000/redmine/

</VirtualHost>
  • If you want to run another domain on you local webserver you need to add a rule to your Hosts file, ie 127.0.0.1 rails.localhost.com
  • All the code required is contained within <VirtualHost> tags
  • This code be used on the whole server or just an individual domain (depends if you define ServerName)

Including the rules in the <VirtualHost> tags is not required but can make handling the code easier and if you do want apply this to a specific domain it is easy to enable it.

My Code

So you have arrived here at the code, just following the simple instructions and edit the files. Redmine will then run at http://localhost/redmine/ on you xampp installation. This does of course assume you have Redmine already running on your Windows PC.

 

Redmine - config.ru

# This file is used by Rack-based servers to start the application.
 
require ::File.expand_path('../config/environment',  __FILE__)
map '/redmine' do
  run RedmineApp::Application
end

Replace the contents of {redmine}/config.ru with this code. This also assumes you have not already made any alterations to this file.

 

Redmine - enviroment.rb 

ActionController::Base.relative_url_root = '/redmine'

Add this to the end of {redmine}config/enviroment.rb. I also think you can put this in either development.rb or production.rb, this change will then only occur in that particular enviroment.

 

Apache / Xampp - httpd.conf

Edit your Apache / Xampp httpd.conf file

LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so

Make sure these modules are enabled in httpd.conf file, they are required.

#################################
# RUBY SETUP
#################################

	# Enforces a trailing slash for the /redmine/ directory
	RedirectMatch ^/redmine$ /redmine/
	RedirectMatch ^/$ /redmine/
	
	# The proxying code
	ProxyRequests Off 
	ProxyPreserveHost On
	ProxyPass /redmine/ http://localhost:3000/redmine/
	ProxyPassReverse /redmine/ http://localhost:3000/redmine/

#################################
# RUBY SETUP
#################################

Add these lines at the bottom of the httpd.conf file.

 

Done

All the changes are now done, you will find that Redmine will be served from http://localhost/redmine/ and http://localhost:3000/redmine/ , this is normal.

If you had your servers running while making these changes you need to restart them both.

You should note that this setup is non-SSL version so is not secure.

General Links

These are the links i used for my research and might be of use.

Redmine / Ruby

Reverse Proxy

Apache

Published in xampp
Read more...
Sunday, 01 February 2015 20:02

Upgrading Redmine

These are my instructions to upgrade your Redmine install on windows and also assumes you have used my instructions for installing redmine, however this is not mandatory but will explain where the filepaths come from.

I will be upgrading Redmine 2.6.1 to Redmine 3.0.0 but i am sure they will be almost the same for other version.

Notes

  • These instructions are based on the official instructions from RedmineUpgrade - Official Instructions.
  • Redmine 3.0.0 now uses mysql 6.1.5
  • the gems are installed to the RubyInstallere and not Redmine

Instructions

Step 1 - Check Requirements

The first step to upgrading Redmine is to check that you meet the requirements for the version you're about to install.

You can easily check what version of Redmine, Ruby and Rails by browsing to Administration/Information and you will see the following information.

Environment:
  Redmine version                2.6.1.stable
  Ruby version                   2.1.5-p273 (2014-11-13) [i386-mingw32]
  Rails version                  3.2.21
  Environment                    production
  Database adapter               Mysql2
SCM:
  Subversion                     1.8.11
  Filesystem                     
Redmine plugins:
  no plugin installed

Redmine 3.0.0 requires, Ruby 1.9.3 or higher (and now supports ruby 2.2).

Step 2 - Backup

  1. Backup your redmine database via phpmyadmin (or any other method you prefer)
  2. make a full backup of your redmine directory (ie D:\Documents\Developer\RubyRoot\redmine\)

Step 3 - Perform the upgrade

    1. If Redmine is running, gracefully stop redmine, the redmine service if running and any related xampp installation (ie D:\Documents\Developer\xampp)
    2. Download the following software
    3. Rename
      D:\Documents\Developer\RubyRoot\redmine\
      to
      D:\Documents\Developer\RubyRoot\old-redmine\
    4. Extract redmine-3.0.0.zip to D:\Documents\Developer\RubyRoot\redmine
    5. From the '.../old-redmine/' directory, copy the following files to the same location in the new redmine installation '.../redmine/'
      • {redmine}/config/database.yml
      • {redmine}config/configuration.yml
      • {redmine}/files/ directory content (this directory contains all your uploaded files so you might not have any).
      • The folders of your custom plugins from your plugins directory (or vendor/plugins directory if upgrading from a 1.x version) into new installation plugins directory. Make sure that you copy only plugins that are were not initially bundled with your previous Redmine setup. My installation does not have any plugins so this step is not needed.
      • If you are running Redmine in a subfolder
        • {redmine}/config/environment.rb (this might require merging rather than copying)
        • {redmine}/config.ru (this might require merging rather than copying)
      • If you have the redmine service copy the following files
        • redmine_service_ctl.rb
        • redmine_service.rb
    6. Open up a command prompt as an administrator and navigate to D:\Documents\Developer\RubyRoot\redmine
    7. Install the required gems by running the following command: 
      bundle install --without development test
      • this installs all the required gems
      • this brings up the following message about the new version of the MySQL connector that has been installed
        ================================================================================

  You've installed the binary version of mysql2.
  It was built using MySQL Connector/C version 6.1.5.
  It's recommended to use the exact same version to avoid potential issues.

  At the time of building this gem, the necessary DLL files were retrieved from:

  http://cdn.mysql.com/Downloads/Connector-C/mysql-connector-c-6.1.5-win32.zip

  This gem *includes* vendor/libmysql.dll with redistribution notice in vendor/README.

================================================================================
      • You need to extract  libmysql.dll from mysql-connector-c-6.1.5-win32.zip. The file is located in the archive at ...\mysql-connector-c-6.1.5-win32\bin
      • Replace D:\Documents\Developer\RailsInstaller\mysql\lib\libmysql.dll with the new version of libmysql.dll
      • you might need to reboot windows for the new version of libmysql.dll to be loaded
    8. Run the following command from your new Redmine root directory: 
      bundle exec rake generate_secret_token
    9. Check for any themes that you may have installed in the public/themes directory. You can copy them over but checking for updated version is ideal.

Step 4 - Update the database

  1. Start your xampp install apache and mysql
  2. This step is the one that could change the contents of your database. Go to your new redmine directory, then migrate your database: 
    bundle exec rake db:migrate RAILS_ENV=production
  3. If you have installed any plugins, you should also run their database migrations: 
    bundle exec rake redmine:plugins:migrate RAILS_ENV=production

Step 5 - Clean Up

  1. Clear the cache and the existing sessions: 
    bundle exec rake tmp:cache:clear tmp:sessions:clear RAILS_ENV=production
  2. Restart the application server (e.g. puma, thin, passenger)
  3. Finally go to "Admin -> Roles & permissions" to check/set permissions for the new features, if any.

Links

Published in Redmine
Read more...
Saturday, 31 January 2015 18:16

Further Redmine Settings

Changing the Logging Level

If you discover that your log file is filling up with lots of uneeded information you can change the logging level from the default.

in the {redmine}/config/additional_enviroment.rb file you can change the logging level. By either adding the line or un-remming one that is present. When you set the error loggin level to error you will only get errors logged to the log file. This will affect both enviroments.

  • config.log_level = : debug
  • config.log_level = : error

The log levels available are: :debug, :info, :warn, :error, :fatal, and :unknown, which correspond to the integers 0-5.

Logging Links

Published in Redmine
Read more...
Sunday, 25 January 2015 17:01

Integrate Subversion with Redmine

Installation of Subversion with redmine can be straight forward if you have some easy toread instructions, so here they are.

There are several levels of integration

Basic SVN Integration

  • browse SVN Repositories
  • close jobs via SVN commit messages
  • compare revisions

Read the official Instructions - RedmineSettings - Repositories

According to the instructions this only works if Redmine and the Subversion Server are on the sam physical machine (this needs 100% clarifying but for now we will assume it is true).

This feature allows you to grab the commits from a predefined SVN Repository (most protocols are supported) and this repository is attached to a project, by doing this when you submit a commit to a repository with a message with correct issue reference syntax, redmine will close the related isse(s) in the project. Git , SVN and other SCM are available. This feature also allows you to perfomr comparisons of different revisions and you eben get the choice to see the changes side-by-side or on top of each other, this feature can be useful depending on how your code looks. There is also a repository browse feature built in to redmine. You can have multiple repositorys per project

Enable Fetch Commits Automatically

In Redmine Administration/Settings/Repositories make sure that Fetch Commits Automatically is ticked. If this option is activated, the application automatically retrieves the new revisions when a user consults the repository.

Official article here

Instructions to add a repository

make sure u use a network version for the repository so you can move it if you want.

  • create a project
  • In the project settings, make sure that the 'Repository' module is enabled and go to the 'Repository' tab.
  • Select the SCM that corresponds to your repository and enter the path or URL of your repository.
  • i used these settings, Redmine was able to download the changes from the repository

SCM Subversion
Main repository checked
identifier test-identifier (dont know what this is for, Repository Identifier)
URL https://svn.exampleserver.com:18080/svn/Test/
user svn-redmine
password xxwhateverxxxx


* when the repository is set you cannot change the details after you have created it except for the surname and password.I have read on the redmine official documentation that you should use a network location for you Subversion repository

  • the first time you view the repository redmine will retrieve all of the changes and import the information

Now that you have configured these settings you can close jobs via a svn commit (see below for forming a correct message for closing jobs) but the draw back is that the changes/commits are only retrieved at certain times

  1. when you browse the SVN repository via Redmine
  2. if you set up a CRON job {add details here}

Allow SVN Commits to Close Jobs

In the step above you configured Redmine to interact with Subversion and to retrieve any changes/commits. Redmine currently parses the commit messages for keywords which reference issue, it does this so it can automatically change these keywords into links. These links are found by using a set of default rules and keywords which can be chamged in the admin panel but the default options should be fine for most people.

The one thing Redmine does not do by default is close issues. You need to specify the keywords that you want to be used as closing keywords and enable the option in the Redmine settings via the admin panel.

Links

What Should i Do?

  1. Browse to Administration/Settings/Repositories/'Referencing and fixing issues in commit messages' Section
  2. Enable 'Allow issues of all the other projects to be referenced and fixed'
  3. Add the following into 'Fixing keywords'
    fix,fixes,fixed,close,closed,patches,patched

    If i get a better list i will add it here, but these keywords give you the gist.

  4. Set 'Applied Status' to Closed
  5. Set '% Done' to 100%
  6. Click Save at the bottom left of the page to save all changes
  7. Done

You can configure whatever rule you want but at least this will get you going. Read the official Documentation for the syntax of closing statements you should use in your commit messages.

NB:

  • I do not know if the keywords are case sensitive

Advanced Keywords

Better Issue Referencing

Redmine has a list of keywords it scans for to identify issues as follows:

refs,references,IssueID

I want to extend that to the following list

refs,references,IssueID,bug,bugs,defect,defects,feature,features,issue,issues,patch

I dont know about whether yoou can close multiple issues but i will add the plural version here anyway

Resolve Issue Keywords (optional)

These are different status and have different uses so require their own settings in the'Referencing and fixing issues in commit messages'.

If you want to have the resolved option set from commit messages you need to do the following:

  1. Browse to Administration/Settings/Repositories/'Referencing and fixing issues in commit messages' Section
  2. Add a new 'Fixing' rule
  3. Add the following into 'Fixing keywords'
    resolve,resolves,resolved
  4. Set 'Applied Status' to Resolved
  5. Set '% Done' to 100%
  6. Click Save at the bottom left of the page to save all changes
  7. Done

Example Commit Message Syntax for Redmine

To reference and issue use a command using any of the issue referencing keywords as below:

issue #1234

To close a job use the following or similiar command with any of the other closing keywords

close #1234

The key thing with these is that that the command is in a certain format

{keyword}{a space}{a hash}{issue number}

Resolved Vs Closed Links

Close Jobs Imediately Upon SVN Submit

To get jobs to close when you actually submit the commit further integration is required which is in the next section.

This is a very useful feature because in my opinion this is how everyone will think this integration works, when you submit a commit with closing statements in the message that issue will be immediately closed. In the 'Basic SVN Integration' this does not happen until you browse the repository or your cron job is fired so the commits get parsed and redmine closes the jobs so there could potentially be a gap between closing the jobs and this apopearing on the Redmine installation.

In this method you basically create a post-commit hook that polls the Redmine installation which then instructs Redmine that some changes have been made and that it should fetch them. It should be noted that this can cause a slight slow down in the commiting of the changes to the repository.

There are 2 types of polls you can use:

  1. Fetches changesets for all active projects

    /sys/fetch_changesets?key=<your service key>

  2. Fetches changesets from the repository of a specific project. This will be quicker that the previous method.

    /sys/fetch_changesets?id=<project identifier>&key=<your service key>

And several delivery methods for the hook:

  1. Visual Basic (.vbs)
  2. wget
  3. cURL
  4. Ruby (via command line)

My Instructions

I will be using cURL for these instructions because it is the easiest solution.

Do this once

  1. Download the correct cURL Binary from here. Get the Generic Non-SSL version for your architecture (x86/x64), security will be addressed in a later article
  2. Copy the file to D:\Documents\developer\SubversionEdge\bin - this folder should be in your path from when you installed Collabnet Subversion Edge. You can put this file anywhere but if it is not in your path you need to use the full location of the executable or add that location to your path.
  3. Disable Fetch Commits Automatically - there is no point in fetching changes upon browsing the repository from within redmine if your changes are always upto date.
    • You make this change by going to Administration/Settings/Repositories/ and untick this option.
  4. Enable Enable WS for repository management - This allows the Webservice to be used for triggering refreshes.
    1. You make this change by going to Administration/Settings/Repositories/ and tick this option.
    2. 'Genereate a Key' for the WS Services
  5. Save Changes

Required For Each Repository

For each repository you need create add an exectuable post-commit hook with the commands in it to trigger Redmine in to checking for changes. I will use a Hook that updates only the repositories in that particular project. It will also be very easy to change to refresh all repositories.

post-commit.bat

@ECHO OFF

curl "http://<redmine url>/sys/fetch_changesets?id=<project identifier>&key=<your service key>"

This is the example code from Redmine's official instructions on calling a URL that will trigger Redmine to refresh all Repositories for that chosen Project.

Create the 'post-commit' Hook

I will assume

  • you have created a test repository called 'Test' and it is physically located at D:\Documents\developer\SubversionEdge\data\repositories\Test
  • Redmine is located at http://localhost/redmine/
  • Your project in Redmine is called 'TestProject'
  • Your WS Key that you generated earlier is 'Password123'
@ECHO OFF

curl "http://localhost/redmine/sys/fetch_changesets?id=TestProject&key=Password123"
  1. create a file called post-commit.bat
  2. edit this file and add the code above in to it and save.
    NB: if the repository you are updating is in another project you will need to alter the project name accordingly, you cannot just copy the Hook file into another repository.

  3. Move this file to D:\Documents\developer\SubversionEdge\data\repositories\Test\hooks
  4. Done

Upon a successful commit to your Subversion Repository, Redmine will check all of the Repositories that belong to the specified project for changes.

You might be asking why cannot i not just update only a specified repository in Redmine rather than all of the repositories that belong to a project. This is a requested feature but for some reason it has not been added yet. Considering things most people will not notice this and if you use the command that refreshes all repositories in Redmine upon a successful commit you will never need this. If you use the 'Refresh All' command you can just copy this to other repository hook folders without alteration.

Useful Links

Automatically Create SVN Repositories

This is the final level of integration but it is not easy and seems to come with its own issues. You have to edit code and do this and that to get it to work. It is quicker to create the SVN Repository in Subversion Edge and then enter the details into redmine as in 'Basic SVN Integration'

I will not be implementing this aas i have easy control over my Subversion Server

See the official page for more information

Published in Redmine
Read more...
Sunday, 25 January 2015 16:59

Configure Redmine as a Windows Service

Installing Redmine as a Service

This can be a tricky thing to get working if you have no knowledge of Ruby or assume that all the scripts you come across actually work. I have found a lot of information about getting Redmine to run as a windows service is quite old and ruby ahs changed a lot since they were written. This article hold all my research from googling around the internet which includes those key niggets of information to help me get the scripts working.

As ever you have come across another article saying how things should work but never just give you the code so you can get Redmine working and worry about other things. This article is diffrerent, i have included all my research so if there are any problems i have not seen or issues that come along later you will be able to fix them withou pulling your hair out and lastly this article has the working scripts. If you just want the scripts so you can get running please scroll to 'The Scripts' and follow the installations instructions and you will be running very quickly.

Notes

General

There are currently a few methods that I have found to create a service:

  1. Mongrel server – seems old way of doing it
  2. Creating a registry entry via a script and then a payload script with the service code
  3. Create the service utilising the windows resource kit by having another program run as a wrapper
  4. Simple – create the service by the command line pointing to your script

 

  • You might need to run this as administrator
  • This creates a service called redmine
  • When created it is set as Manual, you need to change this to automatic - I think!
  • sc delete redmine will delete the service
  • creates a log file in {REDMIN_DIR}\log
  • my code - sc create redmine binPath= "D:\Documents\Developer\RailsInstaller\Ruby2.1.0\bin\rubyw -C D:\Documents\Developer\RubyRoot\redmine\ service.rb"
  • should there be a space at …..redmine\ service.rb
  • these instructions will only set Redmine as a service not all the rails
  • The script of stack overflow don’t forget to change the CONFIG statement, see thread
  • Irb / IRb= iteractive ruby winapi - Ruby: gem win32-service: Create a service with Admin privileges - Stack Overflow
  • notes for the win32-service.gem is in its gem folder under docs which ill give the syntax for using the gem
  • most commands seem to want to run from the project directory - check steps and add reminders where needed.
  • i think where you run the 'rails server' command denotes the root folder of the ruby server
  • unless specified the current directory will be 'c:\windows\system32\'
  • Ruby.exe is the base binary and rubyw.exe is a version that does not present a console which is better because it can run silently.
  • task kill method works if you do not stop the service to early and i need to add the addition of server.pid delete statement
  • unless you change it the default directoy is c:\windows\system32, you can change it on the service command line prompt with the -C option or in the script with the :chdir or Dir.chdir
  • remember running scripts and things from the command prompt will give you more information

When happens when you run the Redmine service

When you start the service it can take up to 60 seconds to properly start and if you look in task manager it is normal to see to instances of ruby either ruby.exe or rubyw.exe

This is what happens:

  1. when you start the windows service this spawns 1 instance of rubyw.exe in windows
  2. the windows service loads the ruby script service.rb and executes it, this the causes antoher instance of rubyw.exe to appear in the windows service list
  3. the webrick server starts creates and populate the /tmp/pid/server.pid with its process id
  4. redmine starts it then replaces the number in server.pid with its own number
  5. servider.pid is created with redmines PID inside (server.pid is the default)
  6. you can see the rubyw.exe is running at about 5% CPU
  7. when the rubyw.exe (with no usage on redmine) drops to 0% the server is running and it is safe to stop the redmine server if required

Errors

When trying to configure redmine to run as a service you might come across some of the issues I am going to outline below but the good thing is I am going to tell you why they occur which will make fixing them stress free.

Error 1053

Windows could not start the redmine service on Local Computer.

Error 1053: The service did not respond to the start or control request in a timely fashion.

error 1053

  • This occurs when there are issues in the script or in particular the binary path definition in the windows service (ie the one you create with the 'sc create' command)
  • When you start the service and it immediately fails it is most likely the binary path statement.
  • If it fails a little way in it is most likely the spawn.process file path is wrong.
  • not pointing to the ruby/rubyw binary correctly (ie full location)
  • not pointing to the script correctely (ie full location)
  • not changing to the script directory (ie -C or :chdir)
  • faulty script, ie written wrong
  • a previously run script has left a server.pid file in the /tmp/pid/ folder which prevents the daemon spawning
  • possibly incompatible builds of win32-services gem

Error 1067

Windows could not stop the redmine service on Local Computer.

Error 1067: The process terminated unexpectedly.

error 1067

  • This can be caused by trying to close the service too quickly after starting it.
  • If you get this message after the service has started correctely and been running for a while and then you stop it the error occurs this is most likely to be you exit code is in correct, perhaps not terminating the correct pid.

Links

Normal Script

Using Windows Resource Kit

Mongrel

Misc Relevant

The Scripts

Simple Script

The simple script is the absolute basic version of the script but does work as expected.

These are my notes on using the script found on redmine.org, HowTo run Redmine as a Windows service (win32-service + taskill approach) - Redmine. This script has issues with it so i have listed them below and then you will find my corrected version which does work.

  • This script will only run once and then you will get an error (1053) everytime you try an run it afterwards - This is because when thescript finishes it does not delete {redmine_dir}/tmp/pids/servere.pid which prevents it from running when ruby does a check for running processes. To fix this i added a line of code at the end to remove the server.
  • when running after correctely setting the details in the script i was still getting a 1053 error. - This is caused because the file location for the spawn.process event needs to be a full address
  • it will give a 1063 error when exiting - This can be caused by the exit! command, so i commented it out

Instructions

  • Install the win32-service Gem. This is done by opening a command prompt (possibly with admin privileges and the internet) and typing: ruby gem install win32-service
  • create a  file called redmine-service.rb putting in the script code below.
  • Place this script the root folder of your redmine installation D:\Documents\Developer\RubyRoot\redmine\
  • In the script update REDMINE_DIR path to fit your Redmine installation
  • open up a command prompt with administrator priviledges and run the following command that will create the Windows Service. Edit the command where needed,  D:\Documents\Developer\RubyRoot\redmine\ is the path of the directory where the service.rb file is located and D:\Documents\Developer\RailsInstaller\Ruby2.1.0\bin\rubyw your Ruby binary.
sc create redmine binPath= "D:\Documents\Developer\RailsInstaller\Ruby2.1.0\bin\rubyw -C D:\Documents\Developer\RubyRoot\redmine\ redmine-service.rb"

When you fill in the command make sure you leave the space inbetween ...\redmine\ redmine-service.rb , the reason for this is that ...\redmine\ beongs with the -C switch which is telling the code to change to the redmine directory before running the script. you can actually add a full path to redmine-service.rb if you wanted to make it easier to read.

  • You can now start and stop the Redmine Service through services.msc or these should work sc start redmine and sc stop redmine
  • If you want to delete the service you can use sc delete redmine

The Script

REDMINE_DIR = 'D:\Documents\Developer\RubyRoot\redmine'
LOG_FILE = "#{REDMINE_DIR}\\log\\redmine-service.log" 

begin
  require 'win32/daemon'
  include Win32

  class RedmineService < Daemon

    def service_init
      File.open(LOG_FILE, 'a'){ |f| f.puts "Initializing service #{Time.now}" }     
	  @server_pid = Process.spawn 'D:\Documents\Developer\RailsInstaller\Ruby2.1.0\bin\rubyw script/rails s -e production', :chdir => REDMINE_DIR, :err => [LOG_FILE, 'a']
    end

    def service_main
      File.open(LOG_FILE, 'a'){ |f| f.puts "Service is running #{Time.now} with pid #{@server_pid}" }
      while running?	  
        sleep 10
      end
    end

    def service_stop
		
		File.open(LOG_FILE, 'a'){ |f| f.puts "Stopping server thread #{Time.now} with pid #{@server_pid}" }  
		system "taskkill /PID #{@server_pid} /T /F"		
		Process.waitall
		File.open(LOG_FILE, 'a'){ |f| f.puts "Service stopped #{Time.now}" }
		pid_file = 'tmp/pids/server.pid'
		File.file?(pid_file) && File.delete(pid_file)
		#exit! # can cause a bad exit		  
    end
	
  end

  RedmineService.mainloop

rescue Exception => e
  File.open(LOG_FILE,'a+'){ |f| f.puts " ***Daemon failure #{Time.now} exception=#{e.inspect}\n#{e.backtrace.join($/)}" }
  raise
end

My Complex Script

This is my script that i have made from several scripts. This is a modern daemon code with logging and a controller file to make install/running/deleting the script as a service. The script will also populate most of the variables if your Ruby path is set and you run the redmine_service_ctl.rb from the root of your redmine installation.

My script might not handle paths with spaces in them correctely. This should be easily fixed by using quotes (") where needed. I might update the scripts if there is any interest.

Instructions

The script configures itself automatically.

  • Install the win32-service Gem. This is done by opening a command prompt (possibly with admin privileges and the internet) and typing: ruby gem install win32-service
  • redmine_service_ctl.rb file
    • create a file called redmine_service_ctl.rb putting the in the Controller Script code below.
    • Place this script the root folder of your redmine installation (eg: D:\Documents\Developer\RubyRoot\redmine\)
    • Edit the file as required
  • redmine_service.rb file
    • create a file called redmine_service.rb putting the in the Daemon / Service Script code below.
    • Place this script the root folder of your redmine installation (eg D:\Documents\Developer\RubyRoot\redmine\)
    • Edit the file as required
  • Install the service - Open up a command prompt with administrator priviledges and run the following command that will create the windows service.

    ruby redmine_service_ctl.rb install

 

If the installation does not work as expected you might need to

  • check your ruby path exists
  • that you have run this script from your redmine root folder
  • the script might automatically run only if your configured MySQL service is already running but this will not affect installtion of the service.
  • Edit the Scripts to insert manuall variables instead of relying on the auto variable code.

 

Controller Script

This script should be called redmine_service_ctl.rb and this is the service management script, it is used to install/stop/start/delete the service from the command line.

############################################################################
#
# redmine_service_ctl.rb
#
# This is a command line script for installing and/or running a small
# Ruby program as a service, in this case it is Redmine.
#
# The service will run Redmine and will record certain events to a log file.
# If enabled, the service can also write a small bit of text
# to the log file every 20 seconds to show it is running.
# Some text will bve written to the log file during the initialization (service_init) step.
#
# It should take about 10 seconds to start, which is intentional - it's a test
# of the service_init hook, so don't be surprised if you see "one moment,
# start pending" about 10 times on the command line.
#
# The Log File is {redmine_dir}/log/test.log.  Feel free to delete it when finished.
#
# To run the service, you must install it first.
#
# Usage: ruby redmine_service_ctl.rb <option>
#
# Note that you *must* pass this program an option
#
# Options:
#    install    - Installs the service.  The service name is "RedmineService"
#                 and the display name is "Redmine Windows Service".
#    start      - Starts the service.  Make sure you stop it at some point or
#                 you will eventually fill up your filesystem!.
#    stop       - Stops the service.
#    pause      - Pauses the service.
#    resume     - Resumes the service.
#    uninstall  - Uninstalls the service.
#    delete     - Same as uninstall.
#
# You can also used the Windows Services GUI to start and stop the service.
#
# To get to the Windows Services GUI just follow:
#    Start -> Control Panel -> Administrative Tools -> Services
#
# by shoulders @ QuantumWarp.com
# used code from 
# http://stackoverflow.com/questions/877943/how-to-configure-a-rails-app-redmine-to-run-as-a-service-on-windows
#
############################################################################

#---------------------------------------------------------------------------
# Includes
#---------------------------------------------------------------------------

	# require 'rubygems' # only for < ruby 1.9
	require 'win32/service'   
	require 'rbconfig'
	include Win32
	include RbConfig  

	# Make sure you're using the version you think you're using.
	puts 'VERSION: ' + Service::VERSION

	SERVICE_NAME 			= 'RedmineService'
	SERVICE_DISPLAY_NAME 	= 'Redmine Windows Service'			
	SERVICE_DESCRIPTION		= 'This is my Windows Redmine Service'
	
#---------------------------------------------------------------------------
# Automatically Create Windows Service Binary Path
#---------------------------------------------------------------------------

	RUBY_BINARY = File.join(CONFIG['bindir'], 'rubyw').tr('/', '\\')
	PATH = ' -C ' + File.dirname(File.expand_path($0)).tr('/', '\\') + '\\ ' + 'redmine_service.rb'
	CMD = RUBY_BINARY + PATH
	# MANUAL OVERIDE
	# This builds the windows binaryPath ="" in the format:
	# CMD = 'D:\Documents\Developer\RailsInstaller\Ruby2.1.0\bin\rubyw -C D:\Documents\Developer\RubyRoot\redmine\ redmine_service.rb'
	# CMD = 'D:\Documents\Developer\RailsInstaller\Ruby2.1.0\bin\rubyw -C D:\Documents\Developer\RubyRoot\redmine\ D:\Documents\Developer\RubyRoot\redmine\redmine_service.rb'
	
	# if you want to uses spaces in your path you need to enclose in each path in quotes  ("")
	# possibly
	# CMD = '"D:\Documents\Developer\RailsInstaller\Ruby2.1.0\bin\rubyw" -C "D:\Documents\Developer\RubyRoot\redmine\ " "redmine_service.rb"'
	# CMD = '"D:\Documents\Developer\RailsInstaller\Ruby2.1.0\bin\rubyw" -C "D:\Documents\Developer\RubyRoot\redmine\" "D:\Documents\Developer\RubyRoot\redmine\redmine_service.rb"'
	
#---------------------------------------------------------------------------
# Perform Service Action Based on Argument supplied
#---------------------------------------------------------------------------
   
# You must provide at least one argument.
raise ArgumentError, 'No argument provided' unless ARGV[0]

# These are the actions performed for the various windows service tasks/functions
case ARGV[0].downcase   

  when 'install'
	 Service.new(
		# These are the service configuration settings that are written to the registry
		# will run automatically without a user being logged in
		:service_name     => SERVICE_NAME,			
		:display_name     => SERVICE_DISPLAY_NAME,			
		:description      => SERVICE_DESCRIPTION,            
		:start_type       => Service::AUTO_START,
		:service_type     => Service::WIN32_OWN_PROCESS,
		:load_order_group => 'Network',			
		:dependencies     => nil,
		#:dependencies     => ['Developer-Apache2.4','Developer-MySQL'],
		#:error_control    => Service::ERROR_NORMAL,
		#:service_start_name => 'SomeDomain\\User',
		#:password         => 'XXXXXXX',
		:binary_path_name => CMD,
	 )
	 puts CMD
	 puts 'Service ' + SERVICE_NAME + ' installed'
	 
  # One of following code sections, which all match to their corresponding 'Windows Service Function' is
  # triggered depending on what argument is supplied in the oommand line.
  
  when 'start'
	 if Service.status(SERVICE_NAME).current_state != 'running'
		Service.start(SERVICE_NAME, nil, 'hello', 'world')
		while Service.status(SERVICE_NAME).current_state != 'running'
		   puts 'One moment...' + Service.status(SERVICE_NAME).current_state
		   sleep 1
		end
		puts 'Service ' + SERVICE_NAME + ' started'
	 else
		puts 'Already running'
	 end
	 
	 
  when 'stop'
	 if Service.status(SERVICE_NAME).current_state != 'stopped'
		Service.stop(SERVICE_NAME)
		while Service.status(SERVICE_NAME).current_state != 'stopped'
		   puts 'One moment...' + Service.status(SERVICE_NAME).current_state
		   sleep 1
		end
		puts 'Service ' + SERVICE_NAME + ' stopped'
	 else
		puts 'Already stopped'
	 end
	 
	 
  when 'uninstall', 'delete'
	 if Service.status(SERVICE_NAME).current_state != 'stopped'
		Service.stop(SERVICE_NAME)
	 end
	 while Service.status(SERVICE_NAME).current_state != 'stopped'
		puts 'One moment...' + Service.status(SERVICE_NAME).current_state
		sleep 1
	 end
	 Service.delete(SERVICE_NAME)
	 puts 'Service ' + SERVICE_NAME + ' deleted'
	 
	 
  when 'pause'
	 if Service.status(SERVICE_NAME).current_state != 'paused'
		Service.pause(SERVICE_NAME)
		while Service.status(SERVICE_NAME).current_state != 'paused'
		   puts 'One moment...' + Service.status(SERVICE_NAME).current_state
		   sleep 1
		end
		puts 'Service ' + SERVICE_NAME + ' paused'
	 else
		puts 'Already paused'
	 end
	 
	 
  when 'resume'
	 if Service.status(SERVICE_NAME).current_state != 'running'
		Service.resume(SERVICE_NAME)
		while Service.status(SERVICE_NAME).current_state != 'running'
		   puts 'One moment...' + Service.status(SERVICE_NAME).current_state
		   sleep 1
		end
		puts 'Service ' + SERVICE_NAME + ' resumed'
	 else
		puts 'Already running'
	 end
	 
	 
  else
	 raise ArgumentError, 'unknown option: ' + ARGV[0]
end

Daemon / Service Script (the actual service)

This should be called redmine_service.rb

############################################################################
#
# redmine_service.rb
#
# This script should automatically configure itself.
# 
# The only thing you might need to alter are the RUBY_BINARY_ARGUMENTS.
# -e is the enviroment and the options are development/production
# -p is the port number to run Redmine on
#
# by shoulders @ QuantumWarp.com
# used code from
# http://stackoverflow.com/questions/877943/how-to-configure-a-rails-app-redmine-to-run-as-a-service-on-windows
#
############################################################################

#---------------------------------------------------------------------------
# Includes
#---------------------------------------------------------------------------

	include RbConfig 

#---------------------------------------------------------------------------
# Automatically Create File Paths
#---------------------------------------------------------------------------

	RUBY_BINARY = File.join(CONFIG['bindir'], 'rubyw').tr('/', '\\')
	#RUBY_BINARY = 'D:\Documents\Developer\RailsInstaller\Ruby2.1.0\bin\rubyw'

	# Redmine 2.x
	#RUBY_BINARY_ARGUMENTS = ' ' + 'script/rails s -e production -p 3000'
	#RUBY_BINARY_ARGUMENTS = ' script/rails s -e production -p 3000'

	# Redmine 3.x
	RUBY_BINARY_ARGUMENTS = ' ' + 'bin/rails s -e production -p 3000'
	#RUBY_BINARY_ARGUMENTS = ' bin/rails s -e production -p 3000'
	
	REDMINE_DIR = File.dirname(File.expand_path($0)).tr('/', '\\')
	#REDMINE_DIR = 'D:\Documents\Developer\RubyRoot\redmine'

	LOG_FILE = "#{REDMINE_DIR}\\log\\redmine-service.log" 
	#LOG_FILE = 'D:\Documents\Developer\RubyRoot\redmine\log\redmine-service.log'

#---------------------------------------------------------------------------
# The Actual Service
#---------------------------------------------------------------------------  

begin  
    
    require 'win32/daemon'
    include Win32

    class RedmineDaemon < Daemon       
        # This method fires off before the +service_main+ mainloop is entered.
        # Any pre-setup code you need to run before your service's mainloop
        # starts should be put here. Otherwise the service might fail with a
        # timeout error when you try to start it.
        #
        def service_init
		
			File.open(LOG_FILE, 'a'){ |f| f.puts "Initializing service #{Time.now.to_s}" }
			
			@server_pid = Process.spawn RUBY_BINARY + RUBY_BINARY_ARGUMENTS, :chdir => REDMINE_DIR, :err => [LOG_FILE, 'a']
			
			# MANUAL COMMANDS
			# Redmine 2.x
			#@server_pid = Process.spawn 'D:\Documents\Developer\RailsInstaller\Ruby2.1.0\bin\rubyw script/rails s -e production -p 3000', :chdir => REDMINE_DIR, :err => [LOG_FILE, 'a']
			# Redmine 3.0
			#@server_pid = Process.spawn 'D:\Documents\Developer\RailsInstaller\Ruby2.1.0\bin\rubyw bin/rails s -e production -p 3000', :chdir => REDMINE_DIR, :err => [LOG_FILE, 'a']
			
        end

        # This is the daemon's mainloop. In other words, whatever runs here
        # is the code that runs while your service is running. Note that the
        # loop is not implicit.
        #
        # You must setup a loop as I've done here with the 'while running?'
        # code, or setup your own loop. Otherwise your service will exit and
        # won't be especially useful.
        #
        # In this particular case, I've setup a loop to append a short message
        # and timestamp to the log file every 20 seconds. Remove the logging
        # when you are using this as a real service otherwise you logfile will
        # become large quickly.
		#
		# not sure what the (*args) means
        def service_main(*args)
						
			File.open(LOG_FILE, 'a'){ |f| f.puts "Service is running #{Time.now} with pid #{@server_pid}" }			
			msg = 'application started at: ' + Time.now.to_s
		
			File.open(LOG_FILE, 'a'){ |f|
				f.puts msg
				f.puts "Args: " + args.join(',')
			}

            # While the daemon is running
            while running?
                if state == RUNNING
                    sleep 20
					
					# write constantly to log file while running for diagnostics only
                    #msg = 'Service is running as of: ' + Time.now.to_s
                    #File.open(LOG_FILE, 'a'){ |f| f.puts msg }
					
                else # PAUSED or IDLE
                    sleep 0.5
                end
            end

            # We've left the loop, the daemon is about to exit.
			
            File.open(LOG_FILE, 'a'){ |f| f.puts "STATE: #{state}" }
            msg = 'service_main left at: ' + Time.now.to_s
            File.open(LOG_FILE, 'a'){ |f| f.puts msg }
			
        end

        # This event triggers when the service receives a signal to stop. I've
        # added an explicit "exit!" here to ensure that the Ruby interpreter exits
        # properly. I use 'exit!' instead of 'exit' because otherwise Ruby will
        # raise a SystemExitError, which I don't want.
        #
        def service_stop
		
			msg = 'Received stop signal at: ' + Time.now.to_s
			File.open(LOG_FILE, 'a'){ |f| f.puts msg }
						
			pid_file = 'tmp/pids/server.pid'
			
			File.open(LOG_FILE, 'a'){ |f| f.puts "Stopping server thread #{Time.now.to_s} with pid #{@server_pid}" }  
			system "taskkill /PID #{@server_pid} /T /F"				
			Process.waitall
			
			File.open(LOG_FILE, 'a'){ |f| f.puts "Service stopped #{Time.now.to_s}" }
			File.file?(pid_file) && File.delete(pid_file)		
			# exit! - what is this exit for? there was a reason, this seems to cause a bad exit with redmine but might be needed for other scripts
		end

        # This event triggers when the service receives a signal to pause. 
        #
        def service_pause
            msg = 'Received pause signal at: ' + Time.now.to_s
            File.open(LOG_FILE, 'a'){ |f| f.puts msg }
        end

        # This event triggers when the service receives a signal to resume
        # from a paused state.
        #
        def service_resume
            msg = 'Received resume signal at: ' + Time.now.to_s
            File.open(LOG_FILE, 'a'){ |f| f.puts msg }
        end
    end

    # Create an instance of the Daemon and put it into a loop. I borrowed the
    # method name 'mainloop' from Tk, btw.
    #
    RedmineDaemon.mainloop

#---------------------------------------------------------------------------
# Service Failure Logging
#--------------------------------------------------------------------------- 

rescue Exception => err
  File.open(LOG_FILE, 'a'){ |fh| fh.puts 'Daemon failure: ' + err + '\n'} # this one line  is from the 1 page script based on demo
  File.open(LOG_FILE,'a+'){ |f| f.puts " ***Daemon failure #{Time.now.to_s} exception=#{err.inspect}\n#{err.backtrace.join($/)}" }
  raise
end

 Notes

  • the ctl file, once run has nothing to do with the daemon. It is just a configurator., the daemon is how indows interacts with ruby
  • this script is written for Redmine 2.x but needs to be slightly amended to run Redmine 3.x . If you dont make the following change, the redmine service will fail to run and you will get the following error in the redmine-service.log file.
    script/rails no longer exists, please use bin/rails instead.

    To fix this you basically need to make the change as told in this error. You will see in the redmine-service.rb i have added Redmine 3.x versions, the lines you need to change from Redmine 2.x to 3.x are line 30 - line 36, the rest should be obvious.

Making Xampp a Dependency for the Redmine Service

If you are using xampp as the donour for the MySQL service you must make sure that this is running when the Redmine service is started otherwise it will fail and not start. To fix this you can make the xampp MySQL (and Apache if required) service as a dependency which means that when you start your redmine service it will start these dependant services first to make sur ethey are running.

Currently the scripts above are not enabled to call Xampp Apache and MySQL services as dependants, the code is there though. There are a couple of steps required to get this working if you want, otherwise you must manually start at least the MySQL service or already have it runnign before you start the redmine service

This feature requires 2 things to be done

  1. Change Xampp Apache and MySQL Service names (Optional)
  2. Edit the Redmine Service Script (we will be using 'My Complex Script')

NB:

  • When the redmine service starts either the Xampp Apache or MySQL then you cannot stop and start these services through the xampp control panel, it just doesnt work and perhaps will get fixed in later versions. These reason is that windows knows that redmine started these services and it should be the redmine service that should stop them which it does when you stop and start it.
  • If you go into windows and stop and start the Apache or MySQL service via services.msc windows is intelligent and knows the redmine service started these services so propmpt you that it will also stop the redmine service if you continue which it will close correctely.
  • If the Apache and MySQL services are already started when you start the redmine service, they will not be classed as dependendant services and can be stoppped and started throught the xampp control panel
  • I find it useful just to run only the  MySQL service as a dependant when playing with the httpd.conf file so i can easily stop and start the apache server in xampp.
  • Redmine only require the requires the MySQL service to be running.

Change Xampp Service Names

If you want to run more than 1 Xampp installation, or even if you dont, i recommend that you should change the default service name for the Xampp Apache and MySQL services on you Developer Xampp instance. The reason for this is that if like me you use Xampp for developing websites on your Windows PC, the Xampp installationscheck by service name,  to see if the Apache and MySQL services are running. This means that if you setup your Developer site with the default service names, when you then open up another instance of Xampp it will think that it is running as a service even if you have not installed that instance as a service.

So by giving different names to these services we will be using for the redmine development xampp it can function independantly to any other xampp instances. This how most people would expect this to work. I also want this seperate installation of xampp because when i am poking around with code or messing databases for my regular web development i do not risk my development enviroment.

Changing the service names is optionals for those budding coders but i definately still recommend it for the reasons outlined above)

Instructions

To change the service names of Apache and MySQL is very easy and requires you to use the xampp control panel.

Apache Service

  1. Open Xampp control panel as an administrator
  2. Make sure the service is not running
  3. If the service has already been installed, uninstall the service
  4. click config (top right)
  5. click 'Service and Port Settings'
  6. click 'Apache' Tab
  7. change the service name
    Apache2.4  ---->  Developer-Apache2.4
  8. Install the service
  9. Done

MySQL Service

  1. Open Xampp control panel as an administrator
  2. Make sure the service is not running
  3. If the service has already been installed, uninstall the service
  4. Click config (top right)
  5. Click 'Service and Port Settings'
  6. Click 'MySQL' tab
  7. Change the service name
    mysql  ---->  Developer-MySQL
  8. Install the service
  9. Done

Edit the Redmine Service Script

This stage is also an easy one and involves editing the redmine_service_ctl.rb from 'My Complex Script' .

  1. Open up a command prompt as an administrator
  2. Navigate to you redmine root directory (should be D:\Documents\developer\RubyRoot\redmine )
  3. If you have installed 'My Complex Script' you need to uninstalled it, run the following command

    ruby redmine_service_ctl.rb uninstall

  4. If you have not installed 'My Complex Script' you need to create the 2 files redmine_service_ctl.rb and redmine_service.rb in your redmine root folder using the appropriate code from above
  5. Open the file redmine_service_ctl.rb in a text editor
  6. Look at line 88 - 89 and edit the file as follows:
    :dependencies     => nil,
#:dependencies     => ['Developer-Apache2.4','Developer-MySQL'],

    You can see that there is a declaration of no dependencies. The line below it defines some depnedecies but is remmed out. Change the lines around so the service has dependencies. The new code should look like the code below, basically you have just changed the poistion of the Hash '#'

    #:dependencies     => nil,
:dependencies     => ['Developer-Apache2.4','Developer-MySQL'],

     As you can see the service names match those i set earler but can be anything as long as they match the appropriate service.

  7. Save the file
  8. Install the redmine service via the command line now that it has dependencies configured. You should still have the command line open in your redmine root directory and running as an administrator

    ruby redmine_service_ctl.rb install

  9. Done

Now start the redmine service and you will find it will start the Xampp apache and MySQL Services automatically.

Redmine as a Windows Service Final Thoughts

This is a robust way of having redmine running on your Windows PC and once configured you can practically forget about it and just use Redmine for what you wanted to use it for in the first place, software development. There are no more settings to do, you are ready to rock and roll.

Published in Redmine
Read more...
Saturday, 17 January 2015 15:47

Install Redmine on Windows

There are 2 different methods to install Redmine on windows and each has their benefits

  Bitnami Redmine StackSeperate Services: Redmine, Edge Subversion and Xampp
Comparison of Redmine Installation Methods
Pros
  • free
  • the install package installs all require assests
  • you can change the different service port numbers
  • redmine is already configured to the defferent services ie subversion
  • easy to install
  • takes abbout 45 mins to install witha few clicks
  • you can selectively install the various different components
    • subversion
    • DevKit
    • PhpMyadmin
    • Redmine
    • Git
  • all services are install as 'Windows Services'
  • free
  • can use ViewVC as an independant difference viewer
  • subversion Edge has an update feature
  • can use other ruby apps in the same install
  • easier to manage subversion with the Webbased Admin panel in Edge Subversion
  • Subversion Edge has a REST API which is good for future 3rd party development
  • Subversion Edge installs its own SSL certificate for SSL communications
Cons
  • upgrading is not a streamlined process
  • control panel has limited functionality meaning you have to edit config settings by hand
  • subversion is via command line only
  • advanced repository refresh via repository hook is not configured
  • singulary focused to running redmine
  • automatic repository does is not configured
  • takes 2+ hours to install
  • some technical knowledge is required
  • you have to manually configure Remine to run as a service

Bitnami Redmine Stack

This by far the easiet method of installing Redmine with subversion integration. You literally only need to click install, select your install location (i would recommend without spaces to be on the safe side), run the installer and then make sure you are happy with the port numbers. Done.

Upgrading the Binami Stack is possible and there are instructions but do require a little bit of planning and as always make sure you have a backup first.

Bitnami also does a Virtual Machine already configured using Linux that will work on all flavours of VMWare and Virtualbox. The homepage is here

Installation Guide For installing Redmine on Windows with Xampp

I searched the internet and did not find that many instructions on how to install Redmine on windows which i found strange. Most of the instructions were complicated and assumed you already had knowledge on how to install Redmine. eventually i cam across the instructions from NBL Creations, but alas, they did not work either. These instructions were set set out in a very straight forward mannor and were writtin in 2010, so i assumed they were correct then but things have changed since then. I proceeded to go through the instructions from NLB and as i cam across issues i found a solutions. This is basically (for now) how my instructions have beend laid out. Where there are issues with the step i have made clear notes on what to do. This is a very easy installation and gives you the benefits of a Xammp installtion with such things as PhpMyadmin.

This is my prefered method of installing Redmine because i can use the installed Ruby services for other development, i get the added benefits of the CollabNet Subversion Edge Server featuers including ViewVC which is an excellent difference viewer. The server identifications are easily changed in the webbased admin which is great when you are playing with the settings to see what you want. It does have its downside that it requires a lot more configuration. It is not all bad becaus ei will also use the xampp server for developing my apps that require a webserver to run.

I would recommend that you read the instructions through once before starting as this process can be quite lenghty.

Software Versions Used

  • Rails Installer 3.1.0 (it is 32bit) (ruby version 2.1.5)
  • ImageMagick-6.7.9-9-Q16-windows-dll.exe (this is 32bit)
  • Redmine 2.6.1
  • mysql-connector-c-noinstall-6.0.2-win32

My Folder Locations I have used (might not be the same in the original instructions)

  • Xampp - D:\Documents\Developer\xampp
  • Railsinstaller - D:\Documents\Developer\Railsinstaller
  • Redmine - D:\Documents\Developer\RubyRoot\redmine
  • Mysql connector - D:\Documents\Developer\Railsinstaller\MySQL
  • Image Magick - D:\Documents\Developer\ImageMagick

Instructions Begin Here

Instructions based on this article - Installing Ruby on Rails and Redmine with XAMPP on Windows 7 - NLB Creations NLB Creations

This takes 2:00hrs to install

  • These instructions assume that you are running an Xampp webserver on you computer.
  • The mysql service needs to be running for Redmine to come up and work
  • do not use spaces in your path to redmine and xampp. There might be ways around this but is not covered here.

I like Xampp. It sets up a full Apache/MySQL/PHP development environment with very little hassle, which means it’s installed on just about every computer I use on a regular basis. I recently had a need to look into Ruby on Rails (specifically so that I could use the Redmine project management web app) and figured it would be easy to integrate it into my existing dev environment.

It kind of… wasn’t.

This is how I finally managed (after two days of ripping my hair out) to do it.

1) Download and install XAMPP.

2) Download RailsInstaller for Windows.

3) Install RailsInstaller to the D:\Documents\Developer\RailsInstaller directory.

- check ‘Install Git (recommended)
- check “Add executables for Ruby, DevKit and Git (if checked above) to the PATH”
– Check “Configure git and ssh when installation has completed” at the end of the install.
– Enter the requested info in the Rails Environment Configuration screen.

- the git thing ask you for your name and email for something to do with GIT / GIT hub. i dont know why you need these, but creates a file in C:\Users\{username}\.gitconfig with these details
- this seems part of the Ruby setup
- if you already have Git information set in the file above, the setup will use that
- restart PC after installing Rails installer (might not be needed)

4) Create a new Ruby on Rails project:

a) Open a cmd window
b) Type: rails new D:/Documents/Developer/RubyRoot/redmine/

- this copies the rails scaffolding/template files into the specified directory.
- Installation might fail because of SSL issues and that files are downloaded on request by this software. There is an updated SSL certificate that was not put in some versions of Ruby. The solution is to manually download the new SSL cert and place in the correct folder before running this command.
ruby on rails - Can't access RubyGems - possibly due to SSL? - Stack Overflow
Download: AddTrustExternalCARoot-2048.pem
put it in the folder - D:\Documents\Developer\RailsInstaller\Ruby2.1.0\lib\ruby\2.1.0\rubygems\ssl_certs
- you might be prompted with
<config/secrets.yml? (enter "h" for help) [Ynaqdh] - i entered 'Y'

5) Add the following to the end of the D:\Documents\Developer\xampp\config\httpd.conf file:
- i could not get this to forward redmine to xampp/apache

# Listen 3000
# LoadModule rewrite_module modules/mod_rewrite.so
#################################
# RUBY SETUP
#################################
<VirtualHost *:3000>

ServerName rails
DocumentRoot "D:/Documents/Developer/RubyRoot/redmine/public/"

<Directory "D:/Documents/Developer/RubyRoot/redmine/">
Options ExecCGI FollowSymLinks
AllowOverride all
Allow from all
Order allow,deny
AddHandler cgi-script .cgi
AddHandler fastcgi-script .fcgi
</Directory>

</VirtualHost>
#################################
# RUBY SETUP
#################################

Side Note: Make sure you try to start/restart Apache from the Xampp control panel after you save the httpd.conf file. If Apache won’t run, try removing the “Listen 3000″ line.

- Just comment out (as in my version) Listen 3000 and LoadModule rewrite_module modules/mod_rewrite.so because if you have apache running on port 3000 the ruby server cannot start and LoadModule rewrite_module modules/mod_rewrite.so is already uncommented in http.conf so is pointless. Might be worth a quick check.
- you need to add 'rails 127.0.0.1' in to your host file
- i also think that <VirtualHost *:3000> should be <VirtualHost *:80> / <VirtualHost *> / <VirtualHost *:81> or whatever your prot number is
- i can't get this to work, it does not seem to do anything except make the http://rails/ domain valid on 127.0.0.1 and this xampp install
- I don’t know if this is needed to be done for httpd-ssl.conf (if you want SSL)

6) Back in the cmd window, navigate to the directory of your project and start the Rails server with the following commands:

cd D:/Documents/Developer/RubyRoot/redmine

          rails server

         - can take a moment to start. The batch file will display information when started

7) In your browser, navigate to http://localhost:3000 where you should see the RoR welcome screen. Once you’ve verified RoR is working, Ctrl-C in the cmd window to shut down the server.

- rails server stops working when you disable the command prompt window

8) Download Redmine and extract the .zip file. Replace the files generated in D:\Documents\Developer\RubyRoot\redmine\ with the files from the .zip file replacing any files when prompted

9) Create your Redmine database. Start Apache and MySql from the Xampp control panel, and in your browser, go to http://localhost/phpmyadmin to access the MySQL interface.

Click the SQL tab and run the following:
create database redmine character set utf8;
create user 'redmineUser'@'localhost' identified by 'myPassword';
grant all privileges on redmine.* to 'redmineUser'@'localhost';

10) Configure Redmine in the redmine/conf/database.yml file by adding your newly created database, username, and password in the appropriate locations.

- the /conf/ folder is now /config/ to redmine/config/database.yml

it should only contain the following

production:
adapter: mysql2
database: redmine
host: localhost
username: root
password: ""
encoding: utf8

- if like me you are running MySQL on another port you also need to add another setting
Info from here - Redmine Wiki - Redmine Install

production:
adapter: mysql2
database: redmine
host: localhost
port: 3307
username: root
password: ""
encoding: utf8


11) In the cmd window, make sure you are still in your Redmine project’s directory, and run the following command:

bundle install

- my install ‘failed’ here and actually failed quite quickly
- I got the error

DL is deprecated, please use Fiddle
Fetching gem metadata from https://rubygems.org/.........
Resolving dependencies...
You have requested:
i18n ~> 0.6.11
The bundle currently has i18n locked at 0.7.0.
Try running `bundle update i18n`

- So I typed ‘bundle update i18n’
- I then got the error

DL is deprecated, please use Fiddle
Fetching gem metadata from https://rubygems.org/.........
Resolving dependencies...
Bundler could not find compatible versions for gem "actionpack":
In snapshot (Gemfile.lock):
actionpack (4.1.8)
In Gemfile:
rails (= 3.2.21) x86-mingw32 depends on
actionpack (= 3.2.21) x86-mingw32
Running `bundle update` will rebuild your snapshot from scratch, using only
the gems in your Gemfile, which may resolve the conflict.

- I then typed ‘bundle update’


12) In all likelihood, it will fail epically when it gets to the rmagick gem (if not, lucky you). Go to http://rubyforge.org/projects/rmagick/ and download the file called rmagick-win32.

a) Install the Ruby Installer DevKit: https://github.com/oneclick/rubyinstaller/wiki/Development-Kit

- the devkit is part of the Railsinstaller so does not need to be downloaded
D:\Documents\Developer\RailsInstaller\DevKit
proceed to next step (b) if DevKit is Present

the configured config.yml contains the following line only: (keep the dash)
          - D:/Documents/Developer/RailsInstaller/Ruby2.1.0

- RubyGems.org Replaces RubyForge as Gem Host - tells you to download gems on RubyGems.org
- HowTo install rmagick gem on Windows - Redmine - this tells of which check boxes to tick
- Installing on Windows · rmagick/rmagick Wiki · GitHub - another rmagik install on windows article and tells the maximum version that works is 6.7.9.9
- Installing RMagick on Windows 7 | Coding with Bunnies - excellent article
- http://rubyinstaller.org/downloads/ - download the devkit here
- if the configuration file will not auto generate you can just edit it manually (see above for setting).
- the configuration file is autogenerated when the command ruby dk.rb init
- Can't get Ruby DevKit configuration file autogenerated properly - Stack Overflow
- Ruby on rails devkit windows - Stack Overflow
- Install Ruby DevKit on Windows 7
- Install Ruby and the Ruby DevKit
- the dev kit needs to match the x32 or x64 of ruby - run 'ruby - v' to see what version
- ruby.rb init - this finds ruby installs and edits the config.yml , this does not always work. if this does not work you need to manuall add the entry
note the dash, it is important, installation of the devkit means, just extract it somewhere
       - D:\Documents\Developer\Ruby\DevKit
- if for some reason you need to manuall install the Devkit, when you extract the devkit to the specified folder, you must not change that folder or name. the 7zip has magic in it and hardcodes the install directory

 

b) Download Imagemagick (check the box to install headers in the installer) : http://www.imagemagick.org/script/binary-releases.php#windows

- make sure that the following are both checked.
“Add Application directory to your system path”
“Install development headers and libraries for C and C++”

- download only the 32-Bit version, installs with x64 will not compile, it will give errors
- the latest version of ImageMagick that will work with rails is 6.7.9.9
- I don’t know the difference netween static or dynamic
- the exact file that I used and worked is ImageMagick-6.7.9-9-Q16-windows-dll.exe
- if 6.7.9.9 will not work another recommendation is 6.7.9.9
- you can get old binaries here: ftp://ftp.sunet.se/pub/multimedia/graphics/ImageMagick/binaries/ImageMagick-6.7.9-9-Q16-windows-x64-dll.exe

c) Install ImageMagick to a location that does NOT contain any spaces in the directory names (like C:\ImageMagick)

- I did D:\Documents\Developer\ImageMagick

d) In the cmd window, run the following:

gem install rmagick --platform=ruby -- --with-opt-lib=c:/ImageMagick/lib --with-opt-include=c:/ImageMagick/include

- is this process creating a GEM (possibly this is compiling the code)
- when installing make sure the 3 following items are checked
- I did gem install rmagick --platform=ruby -- --with-opt-lib=d:/Documents/Developer/ImageMagick/lib --with-opt-include=d:/Documents/Developer/ImageMagick/include

e) Wait. This seems to take an ungodly amount of time, for some reason.

- about 10 – 15 mins on new PC
- it will finish with a “1 gem installed” message

13) Run “bundle install” again.

- my install failed here and actually failed quite quickly
- I got the error

DL is deprecated, please use Fiddle
Fetching gem metadata from https://rubygems.org/.........
Resolving dependencies...
You have requested:
i18n ~> 0.6.11
The bundle currently has i18n locked at 0.7.0.
Try running `bundle update i18n`

- So I typed ‘bundle update i18n’
- I then got the error

DL is deprecated, please use Fiddle
Fetching gem metadata from https://rubygems.org/.........
Resolving dependencies...
Bundler could not find compatible versions for gem "actionpack":
In snapshot (Gemfile.lock):
actionpack (4.1.8)
In Gemfile:
rails (= 3.2.21) x86-mingw32 depends on
actionpack (= 3.2.21) x86-mingw32
Running `bundle update` will rebuild your snapshot from scratch, using only
the gems in your Gemfile, which may resolve the conflict.

- I then typed ‘bundle update’
- I then get the error:

Gem files will remain installed in D:/Documents/Developer/RailsInstaller/Ruby2.1.0/lib/ruby/gems/2.1.0/gems/mysql2-0.3.17 for inspection.
Results logged to D:/Documents/Developer/RailsInstaller/Ruby2.1.0/lib/ruby/gems/2.1.0/extensions/x86-mingw32/2.1.0/mysql2-0.3.17/gem_make.out
An error occurred while installing mysql2 (0.3.17), and Bundler cannot continue.
Make sure that `gem install mysql2 -v '0.3.17'` succeeds before bundling.

14) Manually install the mysql2 gem. For some reason, bundle install doesn’t know how to do this right on Windows.

    1. Download MySQL Connector from mysql.com
    2. Extract the .zip file to D:\Documents\Developer\Railsinstaller\MySQL
    3. Run the following in the cmd window:

      - this command does not work
      gem install mysql2 -- '--with-mysql-lib="D:/Documents/Developer/RailsInstaller/mysql/bin" --with-mysql-include="D:/Documents/Developer/RailsInstaller/mysql/include"'

      - I get the following error
      Gem files will remain installed in D:/Documents/Developer/RailsInstaller/Ruby2.1.0/lib/ruby/gems/2.1.0/gems/mysql2-0.3.17 for inspection.
      Results logged to D:/Documents/Developer/RailsInstaller/Ruby2.1.0/lib/ruby/gems/2.1.0/extensions/x86-mingw32/2.1.0/mysql2-0.3.17/gem_make.out

      - I get this error because the command is wrong. You should use the following command and run it from the redmine project directory
      gem install mysql2 --no-ri --no-rdoc -- --with-mysql-dir=D:\Documents\Developer\RailsInstaller\MySQL

      watch this video(mysql connector)       How to install Ruby on Rails on Windows 7 - Part 3, MySQL connector - YouTube

      - use this file
      mysql-connector-c-noinstall-6.0.2-win32.zip
      Windows (x86, 32-bit), ZIP Archive
      from - http://dev.mysql.com/downloads/connector/c/
    4. Copy libmysql.dll from D:\Documents\Developer\RailsInstaller\mysql\lib to D:\Documents\Developer\RailsInstaller\Ruby2.1.0\bin

15) In the cmd window, execute the following commands:

Make sure that your MySQL server is running before continuing

set RAILS_ENV=production
rake generate_secret_token
rake db:migrate
rake redmine:load_default_data

* That last line is optional. It installs the demo data

- after I run rake generate_secret_token I get an error:
Could not find gem 'selenium-webdriver (>= 0) x86-mingw32' in the gems available on this machine.
Run `bundle install` to install missing gems.

- so I ran ‘bundle install’
- then ‘bundle update i18n’
- then bundle update (perhaps I should of run this in the beginning)

- then re-run rake generate_secret_token
if you still get error messages after successfully installing the the MySQL Connector then you should make sure you have copied the libmysql.dll file to the correct directory and then reboot your computer before continuing.

- back to rake db:migrate (this correctely creates the tables)

 

16) Start the rails server. In the cmd window, navigate to D:\Documents\Developer\RubyRoot\redmine and execute the following:

rails server

17) In your browser, navigate to http://localhost:3000 where you should now see the Redmine homepage.

18) If you’re still sane, enjoy your new Redmine install.

** It’s also worth noting that Redmine won’t actually be RUNNING on XAMPP’s Apache server. It will be running on the Rails server. This just makes it play nice with a new or existing XAMPP install.

    • I need to get redmine to run through apache via the virtual host as it does not work
    • A section – get redmine to run as service automatically
    • A section – configure redmine
    • A section – integrate redmine with subversion ie create repositories and hooks etc.. (can use bitnami for help with files as it is already integrated in the package)

Post Redmine Installation

Once you have completed your Redmine installation there are a few things you might come across or need to do.

 

General Installation Links

Gems

Gems are like plugins for ruby and like a lot of linux systems are downloaded from the internet upon demand rather than being present in your system from the start.

A useful thing is that you can get the different version/revision names from https://rubygems.org/ . If you search this site for your Gem it will give yoou all the previous revisions and actually gives you the install command for that particular version.

Example: gem install win32-service -v 0.8.5
From: https://rubygems.org/gems/win32-service

A good article on Gems, Installing and Uninstalling, RubyMine 7.0.2 Web Help

Installing and uninstalling Gems

This process is failry easy.

To install a gem

Gem uninstall <gem_name>

You can install specific version fof a gem and this can be handy whent here are incompatabilities between vearious gems and software in your install.

If you run the install command with no version number, the latest version of the gem will be installed.

To uninstall a gem

Gem uninstall <gem_name>

When you run uninstall a gem with this command, if there are multiple versions of it, the uninstaller will give you the options of which one to uninstall or all of them.

Published in Redmine
Read more...
Saturday, 10 January 2015 11:10

Integrate Subversion with Mantis

These are my indepth instructions on integrating Mantis with subversion. This came about because i had a website but no subversion server and the easiest way of getting one of those is to install it on my local PC. I then wanted to integrate Subversion with Mantis so when i closed a job it did it automatically. I thought ther would be an easy way of doing this, there is if both the Subversion and Mantis are installed on the same physical machine. I coveer all the integration types here and hopefully fill in a few blanks. I did find the documentation for this whole process lacking. I have scanned the web, categorised these links and figured out the integration for both remote and local integration.

There are a few ways of integrating Matis with a Subversion server and i will outline them below. My instructions will more than likely concentrate on integrating with CollabNet Subversion Edge but i will add some notes on VisualSVN aswell.

  1. checking.php - Mantis Native
  2. Source (Source Control Integration) with SourceSVN plugin
  3. Source with SourceWebSVN
  4. Custom Direct Integrations

For those who want it working and are not to bothered how, if you click this link it will take you straight to my instructions on what i did, no fluff, just 1.2.3...

General Links

Hook Links

Text Notes / Questions

  • what are the 'Enabled Features' in sourceSVN setting?
  • when you submit to a sVN their are hooks that are fired and can run code.
    • in particular there is one called 'post-commit' this code can read the submission notes and act on it if it finds for instance a regexex match
    • curl can then be fired to report this to a bug tracker system which in turn can then read it, and sign of the bug if needed. (CURL might not be used any more)
  • hook folders are in the individual repositoriesand these do not appear until you create the repository/project in SVN admin.
  • check out all regexs for the best one as there a lot of different ones
    • $g_source_control_fixed_regexp = '/\b(?:fixed|fixes|resolved)\s+(?:bug|issue|error)?\s*[#](\d+)\b/i';
    • ?:bug|issue|error|task)(?:feature|bug|issue)
  • can use WebSVn for browsing and integration - but this software is old and might not last long
  • to fully integrate mantis it needs to run on the same server/PC? - check this
  • visualsvn has templates for scripts see this article
  • issues numbers are not linear per project, they are linear over the whole of mantis.
  • remote checkin.php might be possible by SSH
  • The latest version of the hooks are always available in a freshly created repository. These files always have a complete list of what variables are available.

 

 

1 - checkin.php - Mantis Native (Local / Remote)

This is the native basic inbuilt script in Mantis for integrating a Subversion server on the same machine. However with some modification can be used for closing issues from a remote SVN Server.

How Does it work?

The script /mantis/scripts/checkin.php is native to Mantis and 'Provides source control (SCM) integration for closing bugs upon commits'.

This script is used to post infromation about a commit to a remote Subversion Repository. The basic mechanisim is outlined below and assumes the integration is configured correctely.

  1. a user submits a commit to a SVN Repository. With in the commit notes are key notificatsion

    add comment examples here
  2. once the files are commited, the post-commit hook/script of the SVN is triggered and posts the comment to mantis/scripts/checkin.php
    NB: Hooks are specific to individual repositories and not the SVN server
  3. the post is then processed by Mantis with regex rules that have been configured.
  4. if there are any matches Mantis then uses the information to perform actions such as 'close' job (possibly with commit references) as appropriate

This method requires the SVN post-commit hook to have a script on it to run when it is triggered to POST the commit notes to Mantis. Also in mantis you need to add certain settings to the config_inc.php, these settings includes the REGEX settings.

Now a legacy!!

This is now a legacy method of connection your svn server to mantis because it is:

  • old and hard configure with not many options.
  • It requires your Mantis install and Subversion to be on the same server.
  • You must access the script by the command line (cli) because this is how it is written.
  • checkin.php is getting phased out for the plugin SourceSVN which is easier tocnfigure and has more options.
  • SourceSVN, the replacement, can also be used from remote servers as is. SourceSVN also utilises plugins

It should be noted that are some work arounds to allow you to use a remote subversion server with checkin.php and i will make a note of them here to complete my research so you guys dont have to go the hard way to figure all of this out.

Old Read These

Solutions

As mention above there are a couple of ways to get checkin.php to work so i will outline these here.

  1. Standard Method (Local) - Installing the Subversion Server and Mantis on the same server. The following instructions are mainly for linux/unix, but the hook scripts can be easily altered to work in windows and interactions between Xampp and another apache stack is possible

The standard code of Mantis does not allow for web access of the checking script for one reason or another. I have come across 2 unique and clever solutions

  1. PHP Gateway Wrapper (Remote)
  2. Alter Mantis Code (Remote) - as mentioned already mantis checkin.php can only be accessed from the command line, this however is easy to overcome by rewriting the file and once you know this it is a simple procedure.

Research Links:

Generic Subversion

VisualSVN / Windows

 

 

2 - Source Control Integration Plugin (Local / Remote)

The Source Plugin provides a flexible framework for fully integrating any sort of Version Control System (VCS) with Mantis, including support for multiple repositories, changesets or revisions, and file path tracking. It has many features already, such as importing repository history and optional support for remote check-ins or imports, and utilizes a class-based API to access objects in the database.

Source require you to install the Source Core and then an appropriate plugin. There are several plugins and they are available from here... and support several plaforms. Only WebSVN currently has support toa loow to run your own server remotely, the rest are either online services or require Mantis and your Subversion Server to be on the same machine. Beacuse this is a plugin all of the configuration options are handled within the Mantis Plugin Panel, so no code is required to be altered.

The Process Overview

All plugins require a post-commit hook to be configured in the Subversion repository to post information to Mantis.

The checkin / issue close process is simliar to the checkin.php but all the regex settings are configured in the plugin installed in Mantis, the POST URL is different, and the POST only informs Mantis Source that a changes has been made and where to find it, which it then does to aquire all the information whereas the checkin.php method sends all the information in one go to Mantis.

  1. make a change to a file or files
  2. use TortoiseSVN or other client to submit your changes to your Subversion Repository
  3. You start a commit and add your comments including fixes #0000001 and what ever else you want to add to your comment (#0000001 is a phrase that is searched for by manits)
  4. you submit your commit
  5. the post-commit hook is triggered and this hook POSTs that a Revision has occured to Mantis including the REV number and location
  6. Mantis then locally reads the related Subversion Revision
  7. Mantis then pastes the comments in to a note on the related issue.
  8. if the trigger code was a close code ie Fixes #0000001 it will then close the issue, if it was just a reference link ie issue #0000001 it will just post the comment . In both these case, Souce, will do some internal link in Mantis to all the changesets and Diff link - Check this

Different Post Locations for Source

These are some alternative SourceSVN POST URLs that do slightly different things but are not well documented.

http://<your mantis server>/mantis/plugin.php?page=Source/import&id=all&api_key=<your api key>
http://<your mantis server>/mantis/plugin.php?page=Source/import&id=all
http://<your mantis server>/mantis/plugin.php?page=Source/repo_import_latest&id=XX
http://<your mantis server>/mantis/plugin.php?page=Source/index
http://<your mantis server>/mantis/plugin.php?page=Source/checkin

Some of these should have variables with them, via POST or GET (is possibly valid)

Install Source Control Integration Plugin Core

Files can be download here (download the files by clicking 'download zip' on the right). You will also find some simple instructions for installing the plugins.

To get Source to work you need to install its core and then the additional plugin for the platform for you choice. But for now we will just deal with the Source Core

Official Instructions Here (scroll down)

Installation Notes

  • Mantis is required to be on the same machine as the Subversion Server. If like me you are running CollabNet Subversion Edge on your windows PC (if not i recommend it) you will need to run your Mantis install on your PC. The quickest way to do this is to install XAMPP and run it as a service. Make sure you also run it on another port, ie. not 80 or 8080 you will need these for running XAMPP for website development. Using these ports might cause issues with other installed software on your computer.
  • Generate API Key
    • where it says "run openssl rand -hex 12" for us running on windows the binary/exe is here: D:\Documents\Developer\SubversionEdge\bin\openssl.exe
    • open a command prompt and run D:\Documents\Developer\SubversionEdge\bin\openssl.exe rand -hex 12 and copy the results (right click on cmd window, press enter, paste contents into notepad)

Configuration

Most of the options are preconfigured and dont need touching but i will describe what the options are becaus ei have not found descriptions elsewhere. If you need to set any they will be listed at the end.

  • View Threshold -
  • Update Threshold -
  • Manage Threshold -
  • Set Username Threshold -
  • Main Menu Links - show thes link in the Mantis Main Menu
  • Enabled Features -
  • Bug Link Regex Pass 1 - the regex that scans for issue number comments (i.e. issue #0000001)
  • Bug Link Regex Pass 2 - the follow up regex that extracts the actual issue number (i.e. 0000001)
  • Bug Fixed Regex Pass 1 - the regex that scans for fixed issue commnets (i.e. fixed #0000001)
  • Bug Fixed Regex Pass 2 - the follow up regex that extracts the actual issue number (i.e. 0000001)
  • Bug Fixed Status - what status to set when a comment says fixed
  • Bug Fixed Resolution - what outcome to state when issue is fixed
  • Bug Fixed Message Template -
  • Bug Fixed Message View State - should the comment be set as Private or Public
  • Bug Fixed Assign To Committer - the SVN Commiter will be registered as the author of the comment/Issue close
  • API Key - the secret key required for POSTing informatioin to Mantis. It adds a bit of security
  • Allow Remote Check-In (Deprecated) -
  • Allowed Addresses - Allowed addresses for Allow Remote Check-In
  • Allow Remote Imports (Deprecated)
  • Allowed Addresses - Allowed addresses for Allowed Addresses
  • SVN: Path to binary - the physical path to your Subversion Binary. Both windows(c:\csvn\) and Linux paths (/usr/path/bin/) are suitable here.
  • SVN: Command arguments -
  • SVN: Trust All SSL Certs (Requires Subversion 1.6 or newer) - allow to use self-signed certificates and certificates that dont match but are presented.
  • SVN: Use Windows `start` (Requires a configured path to binary) - Is required when using windows

Source Control Integration with SourceSVN Plugin (Local)

This plugin is used to allow direct integration with a local Subversion server (ie on same PC / Machine). It must have direct access to the binaries but can run on either windows or Linux.

Installation

  • From your downloaded package from GitHub, upload the plugin /SourceSVN/ to your {your Mantis}plugins/ folder
  • No Configuration required

Subversion Server Hook

You need to configure a batch file to be run when a commit is made on the Subversion Server. This trigger is called a hook. When you have created your repository there will be a folder inside called Hooks. Each repository gets its own Hooks folder so this process will need to be repeated for each repository.

My instructions are based on me running a CollabNet Subversion Edge Locally on my PC.

  1. navigate to your repository hooks folder (D:\Documents\Developer\SubversionEdge\data\repositories\Test\Hooks\)
  2. create a file called post-commit.bat
  3. paste the contents below from the 'Windows post-commit.bat'
  4. edit the details as needed
  5. done

Linux / Unix post-commit.tmpl

This is an example post-commit script that is found in plugin, /SourceSVN/post-commit.tmpl . It is clearly written for Linux, so will not run on windows.

#!/bin/sh

# Copyright (c) 2012 John Reese
# Licensed under the MIT license

REV="$2"

URL="http://localhost/mantisbt/plugin.php?page=Source/checkin"
PROJECT="Repository Name"
API_KEY="xxxxxxxxxxxxxxxxxxxxxxxxxxx"

LOG_FILE=`mktemp /tmp/svn_${PROJECT}_${REV}_log.XXX`

CURL=/usr/bin/curl

${CURL} -d "repo_name=${PROJECT}" -d "data=${REV}" -d "api_key=${API_KEY}" ${URL} >> ${LOG_FILE}

Windows post-commit.bat

@ECHO OFF

SET REV=%2

SET URL="http://localhost/mantisbt/plugin.php?page=Source/checkin"
SET PROJECT="Repository Name - exactly the same as set in mantis ?"
SET API_KEY="xxxxxxxxxxxxxxxxxxxxxxxxxxx"

LOG_FILE="%PROJECT%_%REV%_log.XXX"

D:\Documents\Developer\SubversionEdge\curl.exe -d "repo_name=%PROJECT%" -d "data=%REV%}" -d "api_key=%API_KEY%" %URL% >> %LOG_FILE%

rem this is better. logging is optional
rem D:\Documents\Developer\SubversionEdge\curl.exe -d "repo_name=%PROJECT%" -d "data=%REV%}" -d "api_key=%API_KEY%" %URL% >> %PROJECT%_%REV%.log

You will need to download the correct version of CURL to allow POSTs from the command line. You can also use wget via Cygwin.

Research Links

These links will include all the basic POST information as well as setup information becasue they are all required for further plugins relating to Subversion on a Local PC.

Generic Subversion

VisualSVN / Windows

 

 

3 - Source Control Integration with SourceSVN Plugin extended by SourceWebSVN Plugin (Local)

WebSVN allows you to show File / Revision differences by using a web browser and this plugin integrates WebSVN. This is an extension to Source Integration and requires the SourceSVN plugin to be installed.

You should install Source Integration Core and SourceSVN and SourceWebSVN. These should already be done from the steps above.

As WebSVN doesnt seem to be developed anymore, there is a replacement Web Based Fifference View called ViewVC. This come preconfigured in CollabNet Subversion Edge. Perhaps the developr could look at making this a plugin and while he is at it consider making it possibly to remotely ccess a Subversion server and which case CollabNet Subversion Edge will be perfect as it has a REST API just for this sort of thing.

Installation Instuctions

  • Install WebSVN and link it to your Subversion Server
  • Install the WebSVN plugin by uploading it to {mantis}/plugins/ folder
  • configure the WebSVN plugin. Use the links below for a more indepth tutorial

Research Links

Generic Subversion

VisualSVN / Windows

 

 

4 - Custom Direct Integrations

This section for for custom integrations such as direct database accessing

Research Links

Generic Subversion

VisualSVN / Windows

  • A nameless soldier...: Integrate Mantis+Subversion on Windows - this tutorial goes through setting up each aspect of the stack using indivdual stack packages and then the guy has used a custom python script to interegate the Mantis SQL database directely. This might be useful on setting up individual stack items in windows

 

 

What I Did

This are the real bare bone instructions of what i did to get the particular methods to work. litterately just what you should to do to get things working. If you want more you should read the rest of the article but i do know people either just want a quick fix or find it easier to process things when it is working.

I have covered the 2 distinct methods of Remote checkin.php via Gateway PHP script (Not fully working) and Remote checkin.php via Altered checkin.php to accept $_POST

 

 

 

Method 1 - Remote checkin.php via Gateway PHP script (Not fully working)

NB: all of this code where appropriate, has logging enabled to help you get this setup, but it is not required for operation

Basics

  • install Mantis on your Web Server
  • install CollabNet Subversion Edge on your windows PC locally. This assumes you have setup the server as per my CollabNet Subversion Server Setup Instructions

Create the Gateway Script

  • Create this script, /mantis/scripts/do_checkin.php
<?php
// For testing if your data is getting POSTed
echo "\r\n";
echo "This is the POSTed data at checkin.php";
echo "\r\n";
echo 'Secret:  ';
echo $_POST['secret'];
echo "\r\n";
echo "\r\n";
echo 'Message:';
echo "\r\n";
echo $_POST['message'];
echo "\r\n";
//exit;

// For diagnostics only
echo 'Connection Made'; echo "\r\n";

// Very cheap security measure, simply so that not just *anybody* can post issues.
if ($_POST['secret'] !== 'somePasswordHere') {
	echo 'You are not Authorised'; echo "\r\n"; // for diagnostics only
	exit;
}

// For diagnostics only
echo 'Authorised, I will now process the Data'; echo "\r\n";

$cmd = 'php checkin.php <<< ' . escapeshellarg($_POST['message']);
system($cmd);

// For diagnostics only
echo 'Data POSTed'; echo "\r\n";

// To prevent infinite looping
exit;

Notes

  • system($cmd); in do_checkin.php i had to put an exit; command after it as it seems to keep re-running do_checkin.php. the POST variables seem not to be passed back from checkin.php. Perhaps using exec() which does not return a result but just executes the command. Not adding the exit; command causes the script to keep re-running and kills my shared server.
  • This needs testing on my local server because you can murder a webserver. It is the looping that does it. try tommorow with exec and see if theyou are not authorised is caused. ie does exec() not loop where as system() loops because it is sending stuff back to the do_plugin.php. either way it is the looping that kills the server and the authorization if statements stops the looping.
  • <<< means it is passing a single string into the file via STDIN/STDOUT , page 404 of the 'The Linux Command Line' PDF/Book

Enable CLi (Command Line) on Web Server

some webservers will has the command line (CLi) disabled. to fix this we need to enabled system(); This is done by the following:

  1. you need to do a dump of the systems php.ini (i assume you can do this) and put it in the php.ini /mantis/scripts/ folder
  2. now in /mantis/scripts/, you need to create a .htaccess file and add the following command in to it:

    SetEnv PHPRC /home/myaccount/public_html/sp/mantis/scripts/php.ini

    NB: you should alter the above location to match your own physical path
    NB: PHPRC does not apply php.ini recusively

  3. now edit the php.ini file
    1. search for disable_functions.
    2. remove system from the disabled functions
    3. save changes

You have now enable CLi for this folder only. if this is a little insecure for you because people will know where to look. You can create a completely random folder any where on your website with a whacky URL and put the do_checkin.php file in there and just alter the file locations appropriately. Then this will also hide this gateway to the regular internet.

Create Subversion post-commit Hook

This hook methods relies on a post-commit.bat being triggered by Subversion which in turn triggers a powershell script which allows us to grab more information about the revision/commit to send this to mantis. Such as real author, time and comments.

post-commit.bat

  • create a post-commit.bat in D:\Documents\Developer\SubversionEdge\data\repositories\Test\hooks\ . You should use the script below and edit as needed
@ECHO OFF

rem For live Subversion Server
rem SET REPOS=%1
rem SET REV=%2

rem For local testing by directly setting variables to use with directely running post-commit.bat
rem saves having to make a change for each submit and the process with that
SET REV="1"
SET REPOS="D:\Documents\Developer\SubversionEdge\data\repositories\Test"

CD %REPOS%\hooks
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy ByPass -File %REPOS%\hooks\post-commit.ps1 "%REPOS%" "%REV%"

This file is what Subversion executes and is where you can change it from live to test. And by Live and Test i mean where you load the variables from. To use in the live mode while testing can be a pain because you have to change a file and submit it with TortoiseSVN everytime you want to use it. Instead i have added the Test mode where you can set the variables REPOS & REV with some arbitrary data allowing you to manually execute post-commit.bat from the command line on your window PC allowing for rapid testing and deployment.

The file above is already configured to be in Test modes, to change it to use the real variables from the Subversion Server (assumes you use a real SVN commit) you need too:

Rem out these Lines

  1. SET REV="1"
  2. SET REPOS="D:\Documents\Developer\SubversionEdge\data\repositories\Test"

UnRem these Lines

  1. rem SET REPOS=%1
  2. rem SET REV=%2

Done, you have now changed the mode. Both these methods will communicate with your website so just bear this in mind.

Powershell Script - post-commit.ps1

  • create a post-commit.ps1 in D:\Documents\Developer\SubversionEdge\data\repositories\Test\hooks\ (one not L). You should use the script below and edit as needed
[System.Reflection.Assembly]::LoadWithPartialName("System.Web") | Out-Null

# output this to send a newline operation to the log file
$newline = "`r`n"

## --------------------------------------------------
## get environmental variables set by Subversion
## --------------------------------------------------

$REPOS = $args[0]
$REV = $args[1]

"$REPOS $REV"

## --------------------------------------------------
## Manually load information from Subversion
## Commit/Rev Author, Date, Time, Comment, Changed Files(s)
## --------------------------------------------------

$svnlook = 'D:\Documents\Developer\SubversionEdge\bin\svnlook.exe'

$auth = & $svnlook author -r $REV $REPOS
$dt = & $svnlook date -r $REV $REPOS
$changed = & $svnlook changed -r $REV $REPOS
$changed = $changed -join "`n"
$log = & $svnlook log -r $REV $REPOS
$log = $log -join "`n"
$data = "Changeset [${REV}] by $auth, $dt
$log
$changed"

## --------------------------------------------------
## POSTing information to checkin.php
## --------------------------------------------------

$postData = "secret=somePasswordHere&message="
$postData += [System.Web.HttpUtility]::UrlEncode($data)

# outputs the post to the logfile (appends)(for diagnostics only)
Write-Output "This is the data POSTed to checkin.php" >> checkin-response.log
$postData >> checkin-response.log

# POSTing using wget and cywin (creates/replaces log with response)
#D:\cygwin64\bin\wget.exe -O checkin-response.log --post-data=$postData http://exampleserver.com/sp/mantis/scripts/do_checkin.php

# POSTing with CURL (exactly the same as wget above)(creates/replaces log with response)(seems slightly slower than wget)
#D:\Documents\Developer\SubversionEdge\curl.exe -d $postData http://exampleserver.com/sp/mantis/scripts/do_checkin.php > checkin-response.log

# POSTing with CURL (same as wget above, but appends log with response)(seems slightly slower than wget)
D:\Documents\Developer\SubversionEdge\curl.exe -d $postData http://exampleserver.com/sp/mantis/scripts/do_checkin.php >> checkin-response.log

## --------------------------------------------------
## POSTing with CURL (alternate syntax)
## --------------------------------------------------

# you must disable $postData and other POSTing method above befor using this
#$secret = 'somePasswordHere'
#$postData = [System.Web.HttpUtility]::UrlEncode($data)

# my prefered CURL call (append output to log file)
#D:\Documents\Developer\SubversionEdge\curl.exe -d secret=$secret -d message=$postData http://exampleserver.com/sp/mantis/scripts/do_checkin.php >> checking-response.log

## CURL Syntax
# Posting variables: 			-d secret=$secret, -d "secret=$secret", -d secret=somePasswordHere, -d $postData,  -d "$postData", -d "message=$postData", -d message=$postData
# Append to log (add at end): 	>> checkin-response.log, >> D:\checkin-response.log
# Replace log (add at end): 	| Out-File checkin-response.log, > checkin-response.log

## --------------------------------------------------

# start a newline in the log file - useful when appending to log file
$newline >> checkin-response.log

# Delete checkin-response.log
#rm checkin-response.log

In this file you will see a lot of code that is not used. I have left this code in there so people, including me, can check syntaxes should they ever wish to make changes because unless you have come across these commands before you could spend quite a lot of time just reseaching the basics.

This file is a Windows Poweshell Script and until i did this project i have never used Powershell. It allows us, via the command line, to use svnlook and get extra information formt he Subversion server that is not passed in the variables we have to work with. We will use this data to populate the close message in the Mantis issue with some more data other than closed. It makes the integration a little tighter.

Errors You might get when running this powershell script (seen best when you manually run post-commit.bat from the command line)

Whilst trying to get the script to work i got these error message, which means more than likely you guys will at some point. So i ahve pasted the errors and their solutions. Simple as that.

D:\Documents\Developer\SubversionEdge\data\repositories\Test\hooks>post-commit.bat
The term 'D:\Documents\Developer\SubversionEdge\data\repositories\Test\hooks\post-commit.ps1 D:\
Documents\edgeSVN\data\repositories\Test ' is not recognized as the name of a c
mdlet, function, script file, or operable program. Check the spelling of the na
me, or if a path was included, verify that the path is correct and try again.
At line:1 char:115
+ D:\Documents\Developer\SubversionEdge\data\repositories\Test"\hooks\post-commit.ps1 "D:\Docume
nts\edgeSVN\data\repositories\Test" " <<<<
    + CategoryInfo          : ObjectNotFound: (D:\Documents\ed...ositories\Tes
   t :String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

most likely the file path or name is wrong

D:\Documents\Developer\SubversionEdge\data\repositories\Test\hooks>post-commit.bat
File D:\Documents\Developer\SubversionEdge\data\repositories\Test\hooks\post-commit.ps1 cannot b
e loaded because the execution of scripts is disabled on this system. Please se
e "get-help about_signing" for more details.
At line:1 char:66
+ D:\Documents\Developer\SubversionEdge\data\repositories\Test\hooks\post-commit.ps1 <<<<  D:\Do
cuments\edgeSVN\data\repositories\Test
    + CategoryInfo          : NotSpecified: (:) [], PSSecurityException
    + FullyQualifiedErrorId : RuntimeException

because i am on windows x64, by default it will not run unsigned code so i added -ExecutionPolicy ByPass -File into the command for powershell

Mantis

These are all of the Mantis related changes you need to do.

Edit Mantis Config File

For some unknown reason you need to add these variables into the config file. They are put there because they become globally available, but i think they should of been there in the first place. These variables are used to match issue number in your comments so it know which issue to close. These can be altered if you know regex to match prefered words ie fixed issue #500.

  • edit config_inc.php and add this code at the bottom.
# Array of allowed SVN hosts that can close issues via curlcheckin.php
$g_source_control_server = '127.0.0.1';


# Account to be used by the source control script.  The account must be enabled
# and must have the appropriate access level to add notes to all issues even
# private ones (DEVELOPER access recommended).
$g_source_control_account = 'svn';

# Checks for 'Issue' references in the comment - used so the comment can be assigned to the correct issue
# Regular expression used to detect issue ids within checkin comments.
# see preg_match_all() documentation at (detect bug or issue in comment)
$g_source_control_regexp = '/(?:bugs?|issues?|reports?)+\s*:?\s+(?:#(?:\d+)[,\.\s]*)+/i';

# Check for 'Fixed Issue' references in the comment - used to id issues that need closing
# Regular expression used to detect the fact that an issue is fixed and extracts
# its issue id.  If there is a match to this regular expression, then the issue
# will be marked as resolved and the resolution will be set to fixed.
$g_source_control_fixed_regexp = '/(?:fixe?d?s?|resolved?s?)+\s*:?\s+(?:#(?:\d+)[,\.\s]*)+/i';


# For open source projects it is expected that the notes be public VS_PUBLIC, however,
# for non-open source it will probably be VS_PRIVATE.
$g_source_control_notes_view_status = VS_PRIVATE;


# If set to a status, then after a checkin with a log message that matches the regular expression in
# $g_source_control_fixed_regexp, the issue status is set to the specified status.  If set to OFF, the
# issue status is not changed. Use CLOSED if you prefer.
$g_source_control_set_status_to = RESOLVED;


# Whenever an issue status is set to $g_source_control_set_status_to, the issue resolution is set to
# the value specified for this configuration.
$g_source_control_set_resolution_to = FIXED;
  • You should note this code is not the same as the code in the articles as the regex was wrong and i made the order more logical. I took the regex code from Source Plugin. There is also furthe regex in the checkincurl.php to take the issue number out of the matches.
  • $g_source_control_server - this feature allows you to set which ip address are allowed to connect. i have this disabled in my curlcheckin.php script because it will only work with dedicated address.
  • $g_source_control_notes_view_status - this requires further codeing (see here) and allows you to set the status of the comment added to the related issue whether you want it private or public.

Configure Mantis

There are some basic settings and obvious stuff to be done in matis.

  • create a user called 'SVN' or whatever you want with 'Developer Access' . This will be the account that is used to post issue closes.
  • create your project. for purposes of this we will call it 'Test'
  • give 'SVN' user permissions to the 'Test' repository if it does not already have

Install CURL / WGET

Obviously you need to install one of these at least to be able to perfomr the POSt from the command line. You dont need both unless you want to play.

  • CURL (easier)
    • download the non SSL version of the package for your system here
    • extract CURL.exe to D:\Documents\Developer\SubversionEdge\
    • when everything is working i would recommend making it SSL. I think all you have to do is swap the binary for one that is SSL (not done it so cant say)
  • wget
    • download the appropriate installer from here
    • install the package to D:\cygwin64\
    • when it mentions about a place to put the downlaoded files it just means where can it create a local download repository because not all packages are installed
    • during the install it will ask you what packages you want
      • search for wget
      • select wget
      • click next
    • done

Make You First Commit

Now this is where things get tricky. I have not been able to make a successful issue close on mantis yet and i believe that is because of the securoty on my web server. If any one manages to get this working i would like to here how.

What i have managed to do is get the Subversion post-commit to POST all the required data to my do_checkin.php successfully.

Below are some responses i get in the log files:

Test Mode set in post-commit.bat

This is the data POSTed to checkin.php
secret=somePasswordHere&message=Changeset+%5b1%5d+by+SYSTEM%2c+2015-01-09+18%3a01%3a24+%2b0000+(Fri%2c+09+Jan+2015)%0d%0aCreating_initial_branch_structure%0d%0aA+++branches%2f%0aA+++tags%2f%0aA+++trunk%2f

This is the POSTed data at checkin.php
Secret:  somePasswordHere

Message:
Changeset [1] by SYSTEM, 2015-01-09 18:01:24 +0000 (Fri, 09 Jan 2015)
Creating_initial_branch_structure
A   branches/
A   tags/
A   trunk/
Connection Made
Authorised, I will now process the Data
Content-type: text/html


This is the POSTed data at checkin.php
Secret:  

Message:

Connection Made
You are not Authorised
Data POSTed

How To Run

  1. Just open cmd
  2. navigate to your post-commit.bat
  3. and run it

see the A lines, this is because no information matches a Revision and so it gets an invalid lookup

Live Mode set in post-commit.bat

This is the data POSTed to checkin.php
secret=somePasswordHere&message=Changeset+%5b54%5d+by+admin%2c+2015-01-17+17%3a14%3a04+%2b0000+(Sat%2c+17+Jan+2015)%0d%0aWe+have+fixed+issue+%230000001+and+solved+some+things+etc.+This+is+a+test+commit%0d%0aU+++trunk%2fchangelog.txt

This is the POSTed data at checkin.php
Secret:  somePasswordHere

Message:
Changeset [54] by admin, 2015-01-17 17:14:04 +0000 (Sat, 17 Jan 2015)
We have fixed issue #0000001 and solved some things etc. This is a test commit
U   trunk/changelog.txt
Connection Made
Authorised, I will now process the Data
Content-type: text/html


This is the POSTed data at checkin.php
Secret:  

Message:

Connection Made
You are not Authorised
Data POSTed

How To Run

  1. Configure TortoiseSVN and checkout your 'Test' repository. Search for my article if i have written it. If not, figure it out.
  2. make a change to a file in your checkedout repositroy, or create a file
  3. right click on the local folder you have used to check out files from you SVN Repository select 'SVN Commit'
  4. TortoiseSVN will open and you should enter a commit message, for now i would recommend

    We have fixed issue #0000001 and solved some things etc. This is a test commit
  5. click submit

Final Thoughts

So this is where i am at. I (with help from alt-tag, GraFX, LinLog) have got all the information to the Mantis checkin.php but cannot get that part of the process to work. All is not lost because with the work i have do here someone else might be able to finish.

If you have a private server or even a server on a local network (seperate mantis and subversion servers) this method might work as is.

 

 

 

 

 

Method 2 - Remote checkin.php via Altered checkin.php to accept $_POST

In this method i have rewritten the checkin.php to accept information via $_POST and will not require CLI access which eludes me, i have also renamed it curlcheckin.php so no core code is hacked.

For further information read the appropriate sections from Remote checkin.php via Gateway PHP script - What i did (Not fully working), this prevents me from duplicating the instructions and having to maintain the text twice.

NB: some of this code where appropriate, has logging enabled to help you get this setup, but it is not required for operation

Basics

  • install Mantis on your Web Server
  • install CollabNet Subversion Edge on your windows PC locally. This assumes you have setup the server as per my CollabNet Subversion Server Setup Instructions

Create Subversion post-commit Hook

use the instructions above for extra information

  • create a post-commit.bat in D:\Documents\Developer\SubversionEdge\data\repositories\Test\hooks\ . You should use the script below and edit as needed - see notes above
  • create a post-commit.ps1 in D:\Documents\Developer\SubversionEdge\data\repositories\Test\hooks\ (one not L). You should use the script below and edit as needed - see notes above but change do_checkin to curlcheckin.php as we will not be using a gateway file.

Create a new checkin.php to accept $_POST

  • create the following file {mantis}/scripts/curlcheckin.php using this code
<?php
# MantisBT - a php based bugtracking system
# Copyright (C) 2000 - 2002  Kenzaburo Ito - kenito@300baud.org
# Copyright (C) 2002 - 2014  MantisBT Team - mantisbt-dev@lists.sourceforge.net
# MantisBT is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
#
# MantisBT is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with MantisBT.  If not, see <http://www.gnu.org/licenses/>.
# See the README and LICENSE files for details

	# --------------------------------------------------------
	# $Id: checkin.php,v 1.5.2.1 2007-10-13 22:35:16 giallu Exp $
	# from https://www.mantisbt.org/bugs/view.php?id=8847
	# Updated 17-01-2015 shoulders @ exampleserver.com
	# --------------------------------------------------------

global $g_bypass_headers;
$g_bypass_headers = 1;
require_once( dirname( dirname( __FILE__ ) ) . DIRECTORY_SEPARATOR . 'core.php' );

// Very cheap security measure, simply so that not just *anybody* can post issues.
if ($_POST['secret'] !== 'somePasswordHere') {
    echo 'You are not Authorised'; echo "\r\n"; // for diagnostics only
    exit;
}

/*# check if we are called from the right IP ( @todo might wanna use a array here )
# Only works for dedicated IP addresses
$t_source_control_server = config_get( 'source_control_server' );
if($_SERVER['REMOTE_ADDR'] != $t_source_control_server )
{
	echo "Not allowed from this IP !!\n";
	exit(0);
}*/

# Check that the username is set and exists
$t_username = config_get( 'source_control_account' );
if( is_blank( $t_username ) || ( user_get_id_by_name( $t_username ) === false ) ) {
	echo "Invalid source control account ('$t_username').\n";
	exit( 1 );
}

# Detect references to issues + concat all lines to have the comment log.
$t_commit_regexp = config_get( 'source_control_regexp' );
$t_commit_fixed_regexp = config_get( 'source_control_fixed_regexp' );

# make sure both the regex scanner and final output gets the full message
$t_line = $_POST['message'];
$t_comment = $_POST['message'];
echo $message;

# Regex Scanning of Comment
$t_issues = array();
$t_fixed_issues = array();

if( preg_match_all( $t_commit_regexp, $t_line, $t_matches ) ) {
	$t_count = count( $t_matches[0] );
	for( $i = 0;$i < $t_count;++$i ) {
		$t_issues[] = $t_matches[0][$i];
	}
}

if( preg_match_all( $t_commit_fixed_regexp, $t_line, $t_matches ) ) {
	$t_count = count( $t_matches[0] );
	for( $i = 0;$i < $t_count;++$i ) {
		$t_fixed_issues[] = $t_matches[0][$i];
	}
}

# If no issues found, then no work to do.
if ( ( count( $t_issues ) == 0 ) && ( count( $t_fixed_issues ) == 0 ) ) {
	echo "Comment does not reference any issues.\n";
	exit(0);
}
		
# If no issues found, then no work to do.
if(( count( $t_issues ) == 0 ) && ( count( $t_fixed_issues ) == 0 ) ) {
	echo "Comment does not reference any issues.\n";
	exit( 0 );
}

# Login as source control user
if( !auth_attempt_script_login( $t_username ) ) {
	echo "Unable to login\n";
	exit( 1 );
}

/*
// not implemented needs additonal coding and $_POST['user'] configuring in the post-commit.bat
# first we try to figure out if we can login with the source control/Subversion username.
# Usernames on Subversion and Mantis should match
$temp_username = user_get_name_by_source_control_id( $_POST['user'] );
if( !auth_attempt_script_login( $temp_username ) ) {
	# Login as source control user
	if ( !auth_attempt_script_login( $t_username ) ) {
		echo "Unable to login\n";
		exit( 1 );
	}	
}
*/

# history parameters are reserved for future use.
$t_history_old_value = '';
$t_history_new_value = '';

# add note to each bug only once
$t_issues = array_unique( $t_issues );
$t_fixed_issues = array_unique( $t_fixed_issues );

# Call the custom function to register the checkin on each issue.
foreach( $t_issues as $t_issue_id ) {
	if( !in_array( $t_issue_id, $t_fixed_issues ) ) {		
		$matches = Null;		
		preg_match('/#?(\\d+)/', $t_issue_id, $matches);
		$t_issue_id  = $matches[1];
		helper_call_custom_function( 'checkin', array( $t_issue_id, $t_comment, $t_history_old_value, $t_history_new_value, false ) );
	}
}
foreach( $t_fixed_issues as $t_issue_id ) {		
		$matches = Null;		
		preg_match('/#?(\\d+)/', $t_issue_id, $matches);
		$t_issue_id  = $matches[1];
		helper_call_custom_function( 'checkin', array( $t_issue_id, $t_comment, $t_history_old_value, $t_history_new_value, true ) );
}

exit( 0 );
  • This file is a rewrite of checkin.php but i have made it accept $_POST and then corrected the regex accordingly.
  • There are lots of different refgex out there but i have used the defaults from Source Integration Plugin because they work.
  • SVN post-commit with windows - Mantis - In source integration you cannot solve muliple jobs, this i a regex fix, also some very good Regex for bug detection. I used this to fix multiple issues not getting processed in checkin.php

Mantis

Follow the instructions above.

Install CURL / WGET

Follow the instructions above.

Make You First Commit

Follow the instructions above.

Final Thoughts

This method is fully working with extended comments and to be honest should of been like from the beginning also native to Mantis. It appears that in the next version of mantis that this methods has been removed. I am sure it will be very easy to add back in and from what i can see it only utilises an inbuilt fuction helper_call_custom_function() to post the comments to the issue and process the issue status.

Although Mantis is moving towards using Source Integration Plugin as its primary integrator, which is good, for those of us who do not have their Subversion Server and mantis on the same server this removal will cause problems.

One way around this is for Mantis / John Reese to add Remote Subversion support by utilising CollabNet Subversion Edge's REST API whilst also adding support for ViewVC which is an excellent Web based Difference viewer that comes preconfigured in this package. It should be noted that ViewVC is availabe as a seperate package.

Most techies that i know use PCs and some have websites. This is probably a large group of people that have PCs and basic websites where they could host Mantis but not a Subversion Server and that is where CollabNet Subversion Edge comes in handy as this setup would be the first suitable arrangement they come across, Mantis on website, Subversion on their own PC.

Published in Subversion (SVN)
Read more...
Friday, 09 January 2015 19:31

CollabNet ViewVC - An Exception Has Occurred

You might come across the following error or one similiar

An Exception Has Occurred
Python Traceback

Traceback (most recent call last):
  File "D:\Documents\Developer\SubversionEdge\lib\viewvc\viewvc.py", line 5068, in main
    request.run_viewvc()
  File "D:\Documents\Developer\SubversionEdge\lib\viewvc\viewvc.py", line 462, in run_viewvc
    self.view_func(self)
  File "D:\Documents\Developer\SubversionEdge\lib\viewvc\viewvc.py", line 2382, in view_roots
    expand_root_parents(request.cfg)
  File "D:\Documents\Developer\SubversionEdge\lib\viewvc\viewvc.py", line 4941, in expand_root_parents
    roots = vclib.svn.expand_root_parent(pp)
  File "D:\Documents\Developer\SubversionEdge\lib\viewvc\vclib\svn\__init__.py", line 76, in expand_root_parent
    roots[rootname] = canonicalize_rootpath(rootpath)
  File "D:\Documents\Developer\SubversionEdge\lib\viewvc\vclib\svn\__init__.py", line 39, in canonicalize_rootpath
    rootpath = _canonicalize_path(rootpath)
  File "D:\Documents\Developer\SubversionEdge\lib\viewvc\vclib\svn\__init__.py", line 23, in _canonicalize_path
    import svn.core
  File "D:\Documents\Developer\SubversionEdge\lib\svn-python\svn\core.py", line 26, in <module>
    from libsvn.core import *
  File "D:\Documents\Developer\SubversionEdge\lib\svn-python\libsvn\core.py", line 25, in <module>
    _core = swig_import_helper()
  File "D:\Documents\Developer\SubversionEdge\lib\svn-python\libsvn\core.py", line 21, in swig_import_helper
    _mod = imp.load_module('_core', fp, pathname, description)
ImportError: DLL load failed: The operating system cannot run %1.

To Fix this you should try the following things (and probably in order)

  1. Stop/Start CollabNet Subversion from within its Web Admin
  2. Reboot your PC
  3. According to several forum threads the presense of files from other installations of programs that require Python, OpenSSL and other things might of lieft files that are causing conflicts in the system
    • libeay32.dll
    • ssleay32.dll
    • py*.dll
  4. Also there could be an issue with enviromental variables incorrectely set or not present

Links

Published in Subversion (SVN)
Read more...
Thursday, 08 January 2015 20:39

My Using Subversion Notes

Notes

  • when you submit to a sVN their are hooks that are fired and can run code.
  • in particular there is one called 'post-commit'that is triggered after a successful commit to the subversion server
  • cURL / wget is used to pass the information via a POST to mantis/redmine
  • hook folder in subversion repository will not appear until you create the repository.
  • revision number is the revisions of the whole project and not just dile/files you have worked on

Links

Client Software

SVN Clients

Plugins for Programs / IDE

  • VisualSVN - Visual Studio add-in for SVN
  • Subclipse - subversion plugin for Eclipse (by CollabNet)
  • AnkhSVN - Visual Studio Plugin (by CollabNet)

Binary Repository Listing Commands

  • svn list --xml "https://<url>"@HEAD --non-interactive --username "some" --password "some_pass" --no-auth-cache --non-interactive --trust-server-cert
  • svn list --xml "https://svn.quantumwarp.com:18080/svn/MyITCRM"@HEAD --non-interactive --username "shoulders" --password "xxxxxxxxxxxxxx" --no-auth-cache --non-interactive --trust-server-cert
  • svn list --xml "https://svn.quantumwarp.com:18080/svn/MyITCRM"@HEAD --no-auth-cache --trust-server-cert --non-interactive
Published in Subversion (SVN)
Read more...
Page 50 of 95