Robot Butler
  Making those complicated tasks nice and easy.
  A hub for guides, walkthroughs and general information.
Follow us on Twitter RSS Feed
Article ImageThis guide assumes that...

A) You're running Ubuntu 12.04.
B) You will only use NginX for MusicBrainz.

This may work on other versions of Ubuntu, but it's possible that package names will change and confusion will creep in so you'll need to make a few changes here and there to suit if required if that is the case.

We will do this whole process as root (so you'll need to sudo su first) and we will be installing MusicBrainz into /opt/musicbrainz to keep everything nice and tidy. The steps are all numbered so you can easily keep track of the process.

1. Install all the packages we will need.
apt-get install postgresql postgresql-server-dev-all postgresql-contrib libpq-dev git-core memcached build-essential libxml2-dev libpq-dev libexpat1-dev libdb-dev libicu-dev nginx perl

2. Now we need to copy the MusicBrainz software to our server.
cd /opt git clone git://github.com/metabrainz/musicbrainz-server.git musicbrainz cd musicbrainz

3. Time to create our preferences file. First we copy the template and then edit it to suit.
cp lib/DBDefs.pm.sample lib/DBDefs.pm

Now edit these specific lines in lib/DBDefs.pm to reflect the following:
sub REPLICATION_TYPE { RT_SLAVE } sub WEB_SERVER { "myserver.com" } sub DB_STAGING_SERVER { 0 } sub CATALYST_DEBUG { 0 } sub DEVELOPMENT_SERVER { 1 } sub EMAIL_BUGS { 'myemail@myserver.com' }
They'll be commented out so you just need to uncomment them and change the parts that don't match what is written above. Don't forget to replace myserver.com with your own domain name.

4. Now we need to create the dependency list of Perl modules. For some reason Musicbrainz doesn't come with one so this has been compiled mostly through trial and error! Paste the following into a file named "cpanfile" in the musicbrainz folder.
requires 'Algorithm::Diff'; requires 'Algorithm::Merge'; requires 'Authen::Passphrase'; requires 'CGI::Expand'; requires 'CSS::Minifier'; requires 'Cache::Memcached'; requires 'Cache::Memcached::Fast'; requires 'Cache::Memory'; requires 'Captcha::reCAPTCHA'; requires 'Catalyst::Action::RenderView' => '0.09'; requires 'Catalyst::Authentication::Credential::HTTP' => '1.009'; requires 'Catalyst::Plugin::Authentication' => '0.10011'; requires 'Catalyst::Plugin::Cache'; requires 'Catalyst::Plugin::Cache::HTTP'; requires 'Catalyst::Plugin::ErrorCatcher'; requires 'Catalyst::Plugin::I18N' => '0.09'; requires 'Catalyst::Plugin::StackTrace'; requires 'Catalyst::Plugin::Unicode::Encoding' => '1.2'; requires 'Catalyst::Runtime' => '5.90007'; requires 'Catalyst::View::TT' => '0.29'; requires 'Class::MOP'; requires 'Clone'; requires 'Cwd'; requires 'DBD::Pg' => '2.11.8'; requires 'DBI' => '1.607'; requires 'DBIx::Connector'; requires 'Data::Compare'; requires 'Data::Dumper::Concise'; requires 'Data::OptList'; requires 'Data::Page' => '2.01'; requires 'Data::UUID::MT' => '0.004'; requires 'Date::Calc' => '5.8'; requires 'DateTime::Format::Duration' => '1.03'; requires 'DateTime::Format::ISO8601'; requires 'DateTime::Format::Natural' => '0.96'; requires 'DateTime::Format::Pg' => '0.16004'; requires 'DateTime::TimeZone'; requires 'Digest::HMAC_SHA1' => '1.03'; requires 'Digest::MD5' => '2.36'; requires 'Digest::MD5::File'; requires 'Digest::SHA1' => '2.110'; requires 'Email::Address'; requires 'Email::MIME'; requires 'Email::MIME::Creator'; requires 'Email::Sender'; requires 'Email::Valid'; requires 'Encode::Detect' => '1.01'; requires 'Exception::Class' => '1.29'; requires 'FCGI'; requires 'FCGI::ProcManager'; requires 'File::Spec'; requires 'GnuPG' => '0.17'; requires 'HTML::FormHandler' => '0.40013'; requires 'HTML::Tiny' => '1.05'; requires 'HTML::TreeBuilder::XPath'; requires 'Hash::Merge'; requires 'IO::All'; requires 'JSON' => '2.14'; requires 'JSON::XS' => '2.27'; requires 'Javascript::Closure'; requires 'List::AllUtils'; requires 'List::MoreUtils' => '0.22'; requires 'List::UtilsBy' => '0.06'; requires 'Locale::Messages'; requires 'Locale::PO'; requires 'Log::Dispatch'; requires 'MRO::Compat' => '0.10'; requires 'Math::Random::Secure'; requires 'Method::Signatures::Simple'; requires 'Module::Pluggable' => '3.9'; requires 'Moose' => '2.0402'; requires 'MooseX::ABC' => '0.05'; requires 'MooseX::Clone'; requires 'MooseX::Getopt'; requires 'MooseX::MethodAttributes' => '0.24'; requires 'MooseX::Role::Parameterized' => '0.16'; requires 'MooseX::Runnable'; requires 'MooseX::Singleton'; requires 'MooseX::Types' => '0.16'; requires 'MooseX::Types::Structured' => '0.24'; requires 'MooseX::Types::URI' => '0.03'; requires 'Net::Amazon::AWSSign'; requires 'Net::Amazon::S3::Policy' => '0.1.2'; requires 'Net::CoverArtArchive'; requires 'Package::Stash' => '0.29'; requires 'Plack::Handler::FCGI'; requires 'Plack::Middleware::Debug'; requires 'REST::Utils' => '0.4'; requires 'Readonly' => '1.03'; requires 'Redis' => '1.961'; requires 'Set::Scalar'; requires 'Statistics::Basic' => '1.6602'; requires 'String::CamelCase'; requires 'String::ShellQuote' => '1.030'; requires 'String::TT' => '0.03'; requires 'Sys::Hostname'; requires 'Template::Plugin::Class'; requires 'Template::Plugin::JavaScript'; requires 'Template::Plugin::Map'; requires 'Template::Plugin::Math'; requires 'Text::MicroTemplate'; requires 'Text::Trim' => '1.02'; requires 'Text::Unaccent'; requires 'Text::WikiFormat' => '0.80'; requires 'Throwable'; requires 'Time::Duration'; requires 'TryCatch'; requires 'URI' => '1.50'; requires 'Unicode::ICU::Collator'; requires 'XML::Generator'; requires 'XML::Parser::Lite' => '0.714'; requires 'XML::RSS::Parser::Lite' => '0.10'; requires 'XML::SemanticDiff' => '1.0000'; requires 'XML::Simple' => '2.14'; requires 'XML::XPath';

5. Carton installs the above packages for us so that's next.
cpan Carton
Answer yes or the default to all (basically just hit enter if you're prompted for anything).

6. Install the Perl packages MusicBrainz needs.
carton install --deployment; carton install; carton install --deployment

7. MusicBrainz needs some special Postgresql modules so lets build and install those.
cd postgresql-musicbrainz-unaccent && make && make install; cd .. cd postgresql-musicbrainz-collate && make && make install; cd ..

8. Now we need to modify Postgres to allow MusicBrainz to log in.
Delete everything in /etc/postgresql/9.1/main/pg_hba.conf and replace it with the following line.
local all all trust
... and then restart Postgres...
service postgresql restart

9. Time to populate the MusicBrainz database with the latest information.
To do this, go to this repostitory and download mbdump-editor.tar.bz2, mbdump-derived.tar.bz2 and mbdump.tar.bz2 to your /opt/musicbrainz folder.

10. Then install them with the following command (this may take a while).
carton exec ./admin/InitDb.pl -- --createdb --import mbdump*.tar.bz2 --echo

11. Now to set up NginX. Open /etc/nginx/nginx.conf and replace the entire file with this content instead.
user www-data; worker_processes 2; pid /var/run/nginx.pid; events { worker_connections 1024; multi_accept on; use epoll; } http { include /etc/nginx/mime.types; default_type application/octet-stream; access_log /var/log/nginx/access.log; error_log /var/log/nginx/error.log; tcp_nodelay on; sendfile on; tcp_nopush on; keepalive_timeout 10; client_max_body_size 20M; client_header_buffer_size 128k; client_body_buffer_size 512k; large_client_header_buffers 4 16k; proxy_buffering on; proxy_buffer_size 4k; proxy_buffers 4 32k; proxy_busy_buffers_size 64k; proxy_temp_file_write_size 64k; fastcgi_buffers 256 4k; fastcgi_read_timeout 5m; gzip on; gzip_min_length 10240; gzip_proxied expired no-cache no-store private auth; gzip_types text/plain text/css text/xml text/javascript application/x-javascript application/xml; include /etc/nginx/conf.d/*.conf; include /etc/nginx/sites-enabled/*; }

12. Now we need to set up our MusicBrainz caching proxy in NginX.
Put the following into /etc/nginx/sites-available/001-musicbrainz.
server { listen 0.0.0.0:5000; server_name myserver.com; access_log /var/log/nginx/001-musicbrainz.access.log; error_log /var/log/nginx/001-musicbrainz.error.log info; location /static/ { root /opt/musicbrainz/root; expires 24h; } location ~* ^/static/.*\.js$ { root /opt/musicbrainz/root; expires max; charset utf-8; } location = /favicon.ico { root /opt/musicbrainz/root; expires 7d; } location = /robots.txt { root /opt/musicbrainz/root; default_type text/plain; try_files $uri $uri.staging expires 1h; } location / { set $script ""; set $path_info $uri; fastcgi_pass 127.0.0.1:55901; fastcgi_param HTTPS $http_x_mb_https; fastcgi_param SCRIPT_NAME $script; fastcgi_param PATH_INFO $path_info; fastcgi_param QUERY_STRING $query_string; fastcgi_param REQUEST_METHOD $request_method; fastcgi_param CONTENT_TYPE $content_type; fastcgi_param CONTENT_LENGTH $content_length; fastcgi_param REQUEST_URI $request_uri; fastcgi_param SERVER_PROTOCOL $server_protocol; fastcgi_param REMOTE_ADDR $remote_addr; fastcgi_param REMOTE_PORT $remote_port; fastcgi_param SERVER_ADDR $server_addr; fastcgi_param SERVER_PORT $server_port; fastcgi_param SERVER_NAME $server_name; fastcgi_param REMOTE_ADDR $http_x_mb_remote_addr; fastcgi_param REMOTE_PORT $remote_port; fastcgi_param SERVER_ADDR $server_addr; fastcgi_param SERVER_PORT $server_port; fastcgi_param SERVER_NAME $server_name; } include /etc/nginx/mbserver-rewrites.conf; }
Don't forget to replace myserver.com with the domain name you're going to use.

13. Now to activate the MusicBrainz proxy and stop NginX taking over port 80
ln -s /etc/nginx/sites-available/001-musicbrainz /etc/nginx/sites-enabled/ ln -s /opt/musicbrainz/admin/nginx/mbserver-rewrites.conf /etc/nginx/ rm /etc/nginx/sites-enabled/default

14. Now we need to keep the system safe and set up MusicBrainz to run as it's own user.
useradd musicbrainz -d /opt/musicbrainz chown -R musicbrainz /opt/musicbrainz

15. Now we create a file that logs all the MusicBrainz output and keeps everything hidden in the background. Add the following lines to /opt/musicbrainz/startmb.sh.
#!/bin/bash echo 'MusicBrainz Starting...' >/var/log/musicbrainz.log exec 2>&1 set -e cd /opt/musicbrainz umask 022 su musicbrainz -c "carton exec -- plackup -Ilib -s FCGI -E deployment --port 55901 --nproc 10 --pid fcgi.pid --keep-stderr=1 >/var/log/musicbrainz.log 2>&1" &

16. Now we need to make sure we can actually run it.
chmod +x /opt/musicbrainz/startmb.sh

17. We need to create this log file so MusicBrainz can write to it.
touch /var/log/musicbrainz.log chown musicbrainz /var/log/musicbrainz.log

18. Finally we start the MusicBrainz software and bring NginX to life to speed things up.
service nginx restart /opt/musicbrainz/startmb.sh

19. Optionally you can also have MusicBrainz start automatically with your server by adding the line...
/opt/musicbrainz/startmb.sh
... into /etc/rc.local before the exit 0 line.

20. To keep Musicbrainz up to date, you need to have the server run the replication packet updates. To do this, simply stick the slave update script into your crontab and set it to run once every hour. The cron line should look something like this:
0 * * * * /opt/musicbrainz/admin/cron/slave.sh

And that's it. You can now test if it's working by going to your domain name followed by :5000 in your web browser, for example http://myserver.com:5000 and you'll see a page that looks something like this.

Article Image


If you see an NginX Gateway Error message then that means NginX is running but MusicBrainz isn't so make sure you've run the MusicBrainz script to start up that side of the service.

If you get an error that a Perl module is missing even though it is installed and it's listed in the cpanfile, you can force install it with the command...
cpanm -Llocal --notest 'Packagename'
So for example to install MooseX::Types you would use...
cpanm -Llocal --notest 'MooseX::Types'
This is because canton, when looking if something is installed, checks your entire system - but when Musicbrainz runs it will only use locally installed modules.

Comments

Auto-Avatar
Joshua Wood   ~   Posted on 2013-03-23 19:04:02
Everything goes great til I get to the importing line. I get this: (cut)
Auto-Avatar
Benjamin Hodgetts   ~   Posted on 2013-03-23 19:13:18
That should be covered by section 7, did you remember to restart Postgres after editing the hba file to allow the user?
Auto-Avatar
Joshua Wood   ~   Posted on 2013-03-23 19:13:43
Figured it out...on your Step 7 "local all all trust" needs to come BEFORE any other permissions. Putting it at the bottom breaks it.
Auto-Avatar
Benjamin Hodgetts   ~   Posted on 2013-03-23 19:14:33
Interesting. I'll edit the guide.
Auto-Avatar
Joshua Wood   ~   Posted on 2013-03-23 22:35:14
As long as I use startmb.sh it updates itself yeah?
Auto-Avatar
Joshua Wood   ~   Posted on 2013-03-23 22:38:34
Do want to say thanks for an excellent guide! Only thing I had to do that isnt mentioned is "apt-get install daemon-tools" for the setuidgid part of startmb.sh
Auto-Avatar
Benjamin Hodgetts   ~   Posted on 2013-03-23 22:50:44
That bit's out of date, I re-wrote it but somehow missed that part. You don't need daemon-tools. Let me update the guide again... and yes, it should keep itself up to date.
Auto-Avatar
Joshua Wood   ~   Posted on 2013-03-23 22:55:31
Do want to say thanks for an excellent guide! Only thing I had to do that isnt mentioned is "apt-get install daemon-tools" for the setuidgid part of startmb.sh
Auto-Avatar
Joshua Wood   ~   Posted on 2013-03-23 22:56:53
./startmb.sh: line 7: exec: setuidgid: not found

thats what I was getting until I installed daemon-tools and it wouldnt start. Running up to date 12.10
Auto-Avatar
Tripkebab   ~   Posted on 2013-03-24 19:49:46
Heya,

I'm currently installing this on Centos5 and have had modify the guide slightly.

I've managed to get as far as step 6 and hitting a problem.

6. MusicBrainz needs some special Postgresql modules so lets build and install those.
cd postgresql-musicbrainz-unaccent && make && make install; cd ..

When running the command im getting an error along the lines of this. Any ideas of what the issue is? Maby a problem with the command itself?


musicbrainz_unaccent.c:8:31: error: tsearch/ts_public.h: No such file or directory
musicbrainz_unaccent.c:9:31: error: tsearch/ts_locale.h: No such file or directory
musicbrainz_unaccent.c:13: warning: data definition has no type or storage class

Additionally in step 5 i'm getting a couple of errors as follows. Could these be the cause of the issues in step 6? Everything else has gone smoothly!

! Configure failed for DBD-Pg-2.19.3. See /root/.cpanm/build.log for details.

99% of the pearl packages installed successfully there are a few exeptions as above.

Any ideas?
Auto-Avatar
deurk   ~   Posted on 2013-03-24 20:47:55
Any reason why in step 1 you put twice the package libpq-dev?
Auto-Avatar
deurk   ~   Posted on 2013-03-24 20:50:34
Oh, and there are 2 sections numbered '2' :)
Auto-Avatar
Benjamin Hodgetts   ~   Posted on 2013-03-24 21:51:23
I really need a forum don't I, haha.

Tripkebab: Sounds like you're missing the postgres development libraries (postgresql-devel or something along those lines, that or it's an incompatible version.

deurk: I wrote this guide on 3 different computers whilst cross-referencing a few things so it was a little muddled. I'll fix that now.
Auto-Avatar
Tripkebab   ~   Posted on 2013-03-24 23:39:53
Thanks I'll check it out. Now go install a forum! :P
Auto-Avatar
deurk   ~   Posted on 2013-03-24 23:47:57
Also had to run twice carton in step 4 to make sure it completed everything (it failed something the first time but went through the second). That might be a good hint to add in there.
Auto-Avatar
Kev   ~   Posted on 2013-04-01 03:01:29
I used the VM version of musicbrainz for several months, that had a status like update where it should how long each request took and what not, is that possible with this?
Auto-Avatar
Kev   ~   Posted on 2013-04-01 03:36:32
I have mine all setup, I can view the musicbrainz site when I got to it in my browser at the IP I set it to. However when headphones tries to pull data for a new group I want to add it shows "retried 3 times, caused by: HTTP Error 500: Internal Server Error.

Again I can go to the same IP in my browser and the musicbrainz site shows just fine.
Auto-Avatar
Benjamin Hodgetts   ~   Posted on 2013-04-01 13:01:16
Kev: Honestly couldn't tell you about the first one. With regards to the second one, you need to visit the URL that Headphones is trying to pull from to test it.
Auto-Avatar
Simon   ~   Posted on 2013-04-07 14:48:18
A sample config for Apache would be great, too.
Auto-Avatar
ScottZ   ~   Posted on 2013-04-26 20:03:44
Step10 Failed for me:
DBD::Pg::st execute failed: ERROR: could not write to hash-join temporary file: No space left on device at /opt/musicbrainz/admin/../lib/Sql.pm line 107, <LOAD> line 70771198.

# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/Internt-root 11G 5.2G 4.9G 52% /


Is this error correct? Will the import use 5G+ ? Now that I have tried this, what is the best method for me to re-attempt?
Auto-Avatar
Benjamin Hodgetts   ~   Posted on 2013-04-26 20:08:24
Import will take up quite a bit of space, so if you're getting "No space left on device" then that's what it'll be.
Auto-Avatar
ScottZ   ~   Posted on 2013-04-28 08:07:17
After I added more space I was able to drop the database and retry step10, it appeared to proceed fine.

You have a typo on step 13
"ln -s /etc/nginx/sites-available/001-musicbrianz /etc/nginx/sites-enabled/"
You mispelled musicbrainz
Auto-Avatar
Benjamin Hodgetts   ~   Posted on 2013-04-28 10:33:39
Whoops, fixed.
Auto-Avatar
Kurt   ~   Posted on 2013-05-20 18:07:58
Hi am following your guide and all has gone well until I try to import the dumps. I run the import command and get the following ....

root@kurt-desktop:/opt/musicbrainz# carton exec ./admin/InitDb.pl -- --createdb --import mbdump*.tar.bz2 --echo
Use of uninitialized value $databaseName in string eq at ./admin/InitDb.pl line 469.
Mon May 20 17:03:33 2013 : InitDb.pl starting
Failed query:
'SELECT 1 FROM pg_shadow WHERE usename = ?'
(musicbrainz)
DBI connect('dbname=template1','postgres',...) failed: FATAL: Peer authentication failed for user "postgres" at local/lib/perl5/DBIx/Connector.pm line 32.

at /opt/musicbrainz/admin/../lib/Sql.pm line 427.
Sql::__ANON__('DBI connect(\'dbname=template1\',\'postgres\',...) failed: FA...') called at local/lib/perl5/Try/Tiny.pm line 100
Try::Tiny::try('CODE(0x144f190)', 'Try::Tiny::Catch=REF(0x24e53f8)') called at /opt/musicbrainz/admin/../lib/Sql.pm line 430
Sql::_select_list('Sql=HASH(0x21fb448)', 'SELECT 1 FROM pg_shadow WHERE usename = ?', 'ARRAY(0x23b5b90)', 'arrayref', 'CODE(0x23b5c38)') called at /opt/musicbrainz/admin/../lib/Sql.pm line 449
Sql::select_list_of_lists('Sql=HASH(0x21fb448)', 'SELECT 1 FROM pg_shadow WHERE usename = ?', 'musicbrainz') called at /opt/musicbrainz/admin/../lib/Sql.pm line 382
Sql::select_single_column_array('Sql=HASH(0x21fb448)', 'SELECT 1 FROM pg_shadow WHERE usename = ?', 'musicbrainz') called at /opt/musicbrainz/admin/../lib/Sql.pm line 397
Sql::select_single_value('Sql=HASH(0x21fb448)', 'SELECT 1 FROM pg_shadow WHERE usename = ?', 'musicbrainz') called at ./admin/InitDb.pl line 191
main::Create('READWRITE') called at ./admin/InitDb.pl line 499
DBI connect('dbname=template1','postgres',...) failed: FATAL: Peer authentication failed for user "postgres" at local/lib/perl5/DBIx/Connector.pm line 32.
at /opt/musicbrainz/admin/../lib/Sql.pm line 429.
Sql::__ANON__('DBI connect(\'dbname=template1\',\'postgres\',...) failed: FA...') called at local/lib/perl5/Try/Tiny.pm line 100
Try::Tiny::try('CODE(0x144f190)', 'Try::Tiny::Catch=REF(0x24e53f8)') called at /opt/musicbrainz/admin/../lib/Sql.pm line 430
Sql::_select_list('Sql=HASH(0x21fb448)', 'SELECT 1 FROM pg_shadow WHERE usename = ?', 'ARRAY(0x23b5b90)', 'arrayref', 'CODE(0x23b5c38)') called at /opt/musicbrainz/admin/../lib/Sql.pm line 449
Sql::select_list_of_lists('Sql=HASH(0x21fb448)', 'SELECT 1 FROM pg_shadow WHERE usename = ?', 'musicbrainz') called at /opt/musicbrainz/admin/../lib/Sql.pm line 382
Sql::select_single_column_array('Sql=HASH(0x21fb448)', 'SELECT 1 FROM pg_shadow WHERE usename = ?', 'musicbrainz') called at /opt/musicbrainz/admin/../lib/Sql.pm line 397
Sql::select_single_value('Sql=HASH(0x21fb448)', 'SELECT 1 FROM pg_shadow WHERE usename = ?', 'musicbrainz') called at ./admin/InitDb.pl line 191
main::Create('READWRITE') called at ./admin/InitDb.pl line 499
Mon May 20 17:03:33 2013 : InitDb.pl failed

Any ideas where i went wrong?
Auto-Avatar
Kurt   ~   Posted on 2013-05-20 18:17:02
Ok I solved that one now when I import im getting the following -

Use of uninitialized value $databaseName in string eq at ./admin/InitDb.pl line 469.
Mon May 20 17:12:43 2013 : InitDb.pl starting
Mon May 20 17:12:43 2013 : Creating database 'musicbrainz_db'
createlang: language "plpgsql" is already installed in database "musicbrainz_db"

Failed to create language -- its likely to be already installed, continuing.
NOTICE: schema "musicbrainz" does not exist
NOTICE: schema "musicbrainz" does not exist
CREATE SCHEMA
CREATE SCHEMA
CREATE SCHEMA
CREATE SCHEMA
Mon May 20 17:12:44 2013 : Installing extensions for PostgreSQL 9.1 or newer (Extensions.sql)
Mon May 20 17:12:44 2013 : Installing musicbrainz_collate.sql extension ... (/usr/share/postgresql/9.1/contrib/musicbrainz_collate.sql)
Mon May 20 17:12:44 2013 : Creating tables ... (CreateTables.sql)
Mon May 20 17:12:44 2013 : Creating tables ... (caa/CreateTables.sql)
Mon May 20 17:12:44 2013 : Creating tables ... (report/CreateTables.sql)
Mon May 20 17:12:44 2013 : Creating statistics tables ... (statistics/CreateTables.sql)
Mon May 20 17:12:45 2013 : Pre-checking mbdump-derived.tar.bz2
Mon May 20 17:12:45 2013 : Schema sequence mismatch - codebase is 16, mbdump-derived.tar.bz2 is 17

Failed to import dataset.
Mon May 20 17:12:45 2013 : InitDb.pl failed
Auto-Avatar
Enverex   ~   Posted on 2013-05-23 17:11:20
You've ended up with mismatched parts. Delete everything and start from scratch is safest.
Auto-Avatar
SWSinRI   ~   Posted on 2013-06-25 01:19:29
I have been trying to set up musicbrainz forever, I create a fresh install of Ubuntu 12.04 LTS Server, update, upgrade, dist-upgrade, and I still get errors at the carton deployment step. Usually that Catalyst::Runtime isn't installed plus a few others.

Am I missing something simple/stupid?
Auto-Avatar
Merson   ~   Posted on 2013-06-25 04:42:03
Followed everything, got no errors. When i go to the webpage i get a 502 error.

Here is the error file:

Error while loading /opt/musicbrainz/app.psgi: Can't locate Plack/Middleware/Debug/Base.pm in @INC (@INC contains: lib local/lib/perl5/i686-linux-gnu-thread-multi-64int local/lib/perl5 . /usr/share/perl/5.14 /usr/lib/perl/5.14) at /usr/share/perl/5.14/parent.pm line 20.
BEGIN failed--compilation aborted at lib/Plack/Middleware/Debug/DAOLogger.pm line 2.
Compilation failed in require at /opt/musicbrainz/app.psgi line 8.
BEGIN failed--compilation aborted at /opt/musicbrainz/app.psgi line 8.

Any help would be appreciated.
Auto-Avatar
Benjamin Hodgetts   ~   Posted on 2013-06-25 11:15:53
You may need to add additional dependencies to the cpanfile manually in the format:

requires "Something";

So for example Plack/Middleware/Debug would be something like:

requires "Plack::Middleware::Debug";

It's a nightmare as Musicbrainz is a moving target, their own documentation is wrong or incomplete and more holes like this keep popping up.
Auto-Avatar
Merson   ~   Posted on 2013-06-25 16:06:16
Thanks that worked. Now i have a new one:

Error while loading /opt/musicbrainz/app.psgi: Can't locate Catalyst.pm in @INC (@INC contains: lib local/lib/perl5/i686-linux-gnu-thread-multi-64int local/lib/perl5 . /usr/share/perl/5.14 /usr/lib/perl/5.14) at local/lib/perl5/i686-linux-gnu-thread-multi-64int/Moose/Exporter.pm line 382.

I've manually installed Catalyst and is showing up to date so i'm unsure what is causing it. These are some of the errors when running the Carton install:

! Installing the dependencies failed: Module 'Catalyst' is not installed
! Bailing out the installation for Catalyst-Plugin-Cache-0.11.

! Installing the dependencies failed: Module 'Catalyst' is not installed
! Bailing out the installation for Catalyst-View-TT-0.40.

! Installing the dependencies failed: Module 'Catalyst' is not installed
! Bailing out the installation for Catalyst-Plugin-Unicode-Encoding-1.8.
Auto-Avatar
Benjamin Hodgetts   ~   Posted on 2013-06-25 16:28:22
And you definitely have the line:

requires "Catalyst";

... in the cpanfile?
Auto-Avatar
Merson   ~   Posted on 2013-06-25 21:41:16
Yes, this is what has totally thrown me off.
Auto-Avatar
Merson   ~   Posted on 2013-06-25 21:47:01
Yes, this is what has totally thrown me off.
Auto-Avatar
Robin   ~   Posted on 2013-07-15 10:35:31
I've had loads of issues with the carton install part. In the end I deleted the carton.lock file (rm carton.lock) and did a carton install. After that the remaining modules installed without problems.
Auto-Avatar
Ruud   ~   Posted on 2013-07-22 20:06:24
Step 2 is missing just a little bit:

git clone --recursive git://github.com/metabrainz/musicbrainz-server.git
cd musicbrainz-server
Auto-Avatar
something   ~   Posted on 2013-07-24 15:09:36
Benjamin, Thanks for the guide. Here is a list of additional steps that I have taken to get this to work which i think has been caused by a change to the musicbrainz package. I noticed your comments on Github and hoping to maybe change your mind. I have done this multiple times on virtual machines running Ubuntu 12.04 to make sure it works.
First thing i did was 'sudo -i' and after step 1 was update the os / packages 'apt-get upgrade' then 'apt-get update' and 'apt-get dist-upgrade'
2. As 'Ruud' has already posted 'git clone --recursive git/github.com/metabrainz/musicbrainz-server.git musicbrainz'
3. Before step 4 'mv carton.lock carton.lock.old'
4. After step 4 'vi /opt/musicbrainz/cpanfile' and add the following lines
requires 'Text::MicroTemplate';
requires 'Plack::Middleware::Debug';
requires 'Catalyst::Plugin::Cache::HTTP';
requires 'Catalyst::Plugin::StackTrace';
requires 'Cache::Memcached::Fast';
requires 'Math::Random::Secure';
requires 'List::AllUtils';
requires 'Set::Scalar';
requires 'Cache::Memory';
requires 'Digest::MD5::File';
5. If anything fails to install after running step 6 then run "rm -rf local" and run command in step 6 again.
6. Before step 10 'su postgres'
7. 'vi /opt/musicbrainz/InitDb.pl' and change READWRITE in line 62 to SYSTEM
8. I cant get the Nginx part to work so I followed the "Keeping Your Local Musicbrainz Server Up-to-Date" section of https://github.com/rembo10/headphones/wiki/Setting-Up-a-Local-Musicbrainz-Mirror

I hope that this helps if you decide to setup a new mirror. If this is difficult to follow then please let me know and I will document from a-z. I have to confirm I am not a Linux guy so the steps I have taken are just to get a musicbrainz mirror operational and not necessarily secure.
Auto-Avatar
Enverex   ~   Posted on 2013-07-24 15:17:01
I'm actually running this on Arch rather than Ubuntu/Debian so I can't actually follow my own guide. I'll try again with your amendments to see if it changes anything but I don't have a massive amount of faith due to their changes.

I'm a Linux Sys Ad so I know my way around the OS, it's the train-wreck that is Musicbrainz mirrors that I'm running into issues with. I've never managed to get it to work outside of a Debian based system anyway, I'm not sure if it's even designed to.
Auto-Avatar
something   ~   Posted on 2013-07-24 15:28:33
Wow quick response. Let me know if you hit a wall. All of it is new to me and to be honest i initially spent days on this. The biggest problems I had were cpan modules not installing and not being able to run the InitDB.pl command. Feel free to email me (hoping you an pull my email address from your site).
Auto-Avatar
Enverex   ~   Posted on 2013-07-24 15:31:09
I was previously able to get everything working up until the actual Plackup server stage (which is the last stage, actually starting the server itself). It was falling over an endless list of dependencies that eventually I couldn't fill in the holes for.

I'll let you know how it goes from this point.
Auto-Avatar
something   ~   Posted on 2013-07-24 15:44:29
Also came across that issue. They are the dependencies in my step 4. Every time I tried to start the server it told me i was missing blah so I would add blah to cpanfile and run 'carton install --deployment' again, then the next missing file so do the same process. I hit a wall when one of the dependancies wouldn't install so i ran 'rm -rf local' and then 'carton install --deployment' which seemed to correct the issue. I am pretty sure that I have listed them all in my step 4. I was left wondering if it would ever end.
Auto-Avatar
Kurt   ~   Posted on 2013-07-29 21:06:00
Hi something,
Would love to see a step by step guide to getting this going, followed Envermexs guide and I could never get all the dependencies to install :-(
It would have to be an idiot proof a-z as I'm not a linux guru by any means lol
Auto-Avatar
Enverex   ~   Posted on 2013-07-30 09:14:33
something: Still failing after your amendments with more missing dependencies. I'll fire-fight those now and see how far I get.

Ok, I've set up a topic for this discussion now as I think it's a bit harder to track this with comments. Feel free to continue the discussion here - https://xnode.org/forum/discussion/2/setting-up-a-musicbrainz-mirror-in-ubuntu-continuation
Auto-Avatar
Enverex   ~   Posted on 2013-07-30 10:11:41
Alright, I finally got my installation working. I've re-done the problematic parts of the guide and added in a force method to fix other dependency issues.
Auto-Avatar
something   ~   Posted on 2013-07-30 14:44:53
Kurt: see how you go with Enverex's instructions. Unlike me he is actually a Linux sysadmin where as I am the opposite (Windows sysadmin). If you have issues then post them up on Enverex's new forum which he has linked above and we can go from there. The best piece of advise I can give you is if you are serious about doing this Google and persistence are going to be your freind's here. As Enverex has stated on github the Musicbrainz admins are forever making changes and not really managing the documentation. I think that once you get a musicbrainz mirror running then you are good unless something happens with the server hosting it or there is a major update.

Enverex: Nice work on getting it going again. I am not surprised about missing dependencies. Probably the best addition to these instructions is the force fix dependency issues. I will follow your instructions over the next couple of days and post back on the forum as to how it goes. I am sure that there are a lot of thankful people out there who appreciate you bringing your mirror back to life.
Auto-Avatar
Enverex   ~   Posted on 2013-07-30 15:27:26
Mirror should be back up soon, I'm having to reimport all the Musicbrainz archive data as it was horribly out of date.
Auto-Avatar
Kurt   ~   Posted on 2013-08-13 06:15:00
Am getting this after running cpan Carton

Fetching with HTTP::Tiny:
http://mirror.tje.me.uk/pub/mirrors/ftp.cpan.org/modules/03modlist.data.gz
Going to read '/root/.cpan/sources/modules/03modlist.data.gz'
............................................................................DONE
Going to write /root/.cpan/Metadata
Running make for E/ET/ETHER/local-lib-1.008011.tar.gz
Fetching with HTTP::Tiny:
http://mirror.tje.me.uk/pub/mirrors/ftp.cpan.org/authors/id/E/ET/ETHER/local-lib-1.008011.tar.gz
Fetching with HTTP::Tiny:
http://mirror.tje.me.uk/pub/mirrors/ftp.cpan.org/authors/id/E/ET/ETHER/CHECKSUMS
Checksum for /root/.cpan/sources/authors/id/E/ET/ETHER/local-lib-1.008011.tar.gz ok
---- Unsatisfied dependencies detected during ----
---- ETHER/local-lib-1.008011.tar.gz ----
ExtUtils::MakeMaker [build_requires]
Running make test
Make had some problems, won't test
Delayed until after prerequisites
Running make install
Make had some problems, won't install
Delayed until after prerequisites
Can't locate local/lib.pm in @INC (@INC contains: /root/perl5/lib/perl5 /etc/perl /usr/local/lib/perl/5.14.2 /usr/local/share/perl/5.14.2 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.14 /usr/share/perl/5.14 /usr/local/lib/site_perl /opt/musicbrainz-server) at /usr/share/perl/5.14/CPAN/FirstTime.pm line 1300.
root@kurt-lounge-pc:/opt/musicbrainz-server#

Is this correct?

PS cant add comments on https://xnode.org/forum/discussion/2/setting-up-a-musicbrainz-mirror-in-ubuntu-continuation as it wont send me the activation email !
Auto-Avatar
Enverex   ~   Posted on 2013-08-14 14:06:35
It is sending the email, but Hotmail are blocking it because apparently this server is "on their blacklist" despite the fact that I think I can count the amount of emails it has ever sent to Hotmail on one hand. I'd recommend changing to a reliable mail provider as (working for a hosting company) I know they are banning massive ranges of mail servers these days for literally no reason.

Anyway, make sure you have at least the base Perl stuff installed as it looks like you may be missing some critical base dependencies. Can't say I've seen that one before though.
Auto-Avatar
lewis   ~   Posted on 2013-10-11 14:06:22
FYI they have change how they start the server. carton exec -- plackup -Ilib -r to plackup -Ilib -r
Auto-Avatar
Vince   ~   Posted on 2013-11-17 11:53:18
Something else I have found is required now is to install redis-server using apt-get otherwise you end up with errors trying to view the pages. No extra configuration of this is required though.
Auto-Avatar
Brian   ~   Posted on 2014-06-06 04:44:14
When attempting to start, I get an error "carton: command not found". I tried switching to the musicbrainz server and running carton with no luck. I am now attempting to install local::lib and then use cpanm to install carton for the musicbrainz user, but not having much luck yet.

How are you getting the musicbrainz user access to carton?
Auto-Avatar
Enverex   ~   Posted on 2014-06-06 10:12:17
Unfortunately I'm not, I've given up on this. They keep changing the software and don't document how it should work making it almost impossible to get it working. Their seemingly only supported method is using their pre-created virtual image.
Auto-Avatar
Ian   ~   Posted on 2014-06-23 20:00:06
Unfortunately the pre-created virtual image has a big flaw:
The provided disk size is too small if you try "bin/reindex" to create the local search index. You will get - after wasting some hours - a "...No space left on device".

The developers should really care about such problems, but they seem not to be interested in this mirror/offline solutions.
Auto-Avatar
chirlu   ~   Posted on 2014-11-24 03:14:46
MusicBrainz has abandoned carton and now (since quite a while, actually) recommends using local::lib instead. People following the “official” installation notes (INSTALL.md in the MusicBrainz repository) have mostly been successful, judging from feedback on IRC and the forums.
Auto-Avatar
Enverex   ~   Posted on 2014-11-24 10:33:07
That's good to hear. The previous installation process was a nightmare. Hopefully it's streamlined now and considerably less error-filled!

Post a Comment

    Name
    Email (not displayed)

The name of this website is Robot...      


Info STATISTICS

  • 14 categories
  • 50 articles
  • 137 comments

Site, design and code by Benjamin Hodgetts.