Andromeda/Public Server Transfer

All of nottinghack.org.uk will move to a new server, this involves quite a lot of work!

Below is an ordered plan of action

Server Build

Buy the new server, build the OS, install required services (sshd, web server, mysql, etc)

Plan is to base this server on nginx, depending on testing.

Quick list of the planed setup ('RepRap' Matt (talk))

• ClonDNS (to handle all DNS)
• Linode 2gb (with Linode Backup)
• Jungle Disk ( Backups, To cloud files or S3, not sure which yet)
• Debian Wheezy
• IPv4 & IPv6 Enabled and Firewalled
• Ttckeeper (we log all changes to /etc, in a git repo so we know who to blame)
• Nginx (1.4.2 dotdeb maintained)
• PHP5-FPM (5.4 dotbeb maintained)
• Mysql (5.5, wheezy core)
• Postfix
• Dovecot
• OpenDKIM
• RoundCube
• ViMbAdmin (postfix webadmin, i think this will work for us)
• phpMyadmin
• new work press
• new mediawiki
• move content* move email* take backups

Linode Stack Script

Linode provide a the ability to run stackscripts on servers, these are great for doing auto deployment. I've been working on one for the new server that will get the following stuff setup: --'RepRap' Matt (talk)

• system update
• install basic support programs and utilities that no linux should be with our
• setup etckeeper versioning on /etc
• grab config bundle
• time zone ( we are going UTC)
• static ip address
• hostname andromeda.lwk.me
• ditch un-needed dhcp
• iptables and ip6tables
• lockdown ssh config
• add users dpslwk daniel james lwk nottinghack, with shh keys and lock out root
• add dotdeb repo
• install mysql
• setup mysql users
• install php5-fpm and extras
• install nginx
• install postfix, dovecot, opendkim
• ViMbAdmin support packages
• remove exim
• copy across configs
  • mysql
  • ssl certs?
  • nginx
  • php5-fpm
  • import mail db
  • postfix
  • dovecot
  • openDKIM
  • fail2ban
  • vimbadmin
• clone/install vimbadmin
• base dir setup for sites
• php pool setup
• nginx sites

Post Stack Script

After the stack script there will still be a number of things to check off, install and setup by hand before we are ready for the main site moves.

• check ip's
• check hostname/staic ip/resolve/hosts
• check iptables ip6tables
• install jungle disk
• setup jungle disk backup's
• mysql db backup script

Every thing should now be in place to do the big moves detailed below

Other bits to do after the main move

• roundcube install
• ZNC irc bouncer
  • install
  • xfer config
  • update firewall configs and restore
• APC stats page
• 'RepRap' Matt other sites

Web Root

Before most of the remaining changes can be made, the server will need to have a live webroot. Maybe we can repurpose nottinghack.co.uk temporarily, or register a new domain. Called <new-domain> in steps below.

All the steps below assume that the webroot is in the backup schedule

New Folder layout

The current basic layout of the nottinghack home folder is this:

/home/nottinghack/+
                  |-planet/                 < Planet back end
                  |-public_html/+           < Wordpress install
                  |             |-members/  < members guide pdf (not sure if this is still needed??)
                  |             |-p/        < Planet html files
                  |             |-w/        < Mediawiki install
                  |-www_secure/             < secure web files (dm2tweet)

But looking to move to something a little more separated:

/home/nottinghack/+
                  |-planet/                 < Planet back end
                  |-public_html/+             < Wordpress install
                  |           |-members/    < members guide pdf (not sure if this is still needed??)
                  |-public_planet/          < Planet html files
                  |-public_wiki/            < Mediawiki install
                  |-www_secure/             < secure web files (dm2tweet, Wordpress config, Mediawiki config)

I like this, but seeing as public_wp will actually just be the main site and include things like dm2tweet too, why don't we conform a little and call it public_html? James (talk) 02:18, 21 August 2013 (EST)
Happy to stick with publi_html --'RepRap' Matt (talk) 04:45, 22 August 2013 (EST)

Databaes

On the current server there is just one DB for WP and MW
New config will have three.

nh        < general
nh-wp     < wordpress only
nh-wiki   < media wiki only

Important design is separate DB's for WP and MW, but have also provisioned a third shared DB for other small things that might need it (planet, dm2tweet, instrumentation, HMS.... etc)

Each will have a it's one user, with the same name as the DB.

Planet

Easiest to move, so let's move it first. No actual data is involved.

Files

• Transfer the planet generation files outside of the webroot.
• Create necessary files underneath webroot (call folder "p")
• run generator to check that it is pulling in all blogs.

Set up

• Set "public_planet" as seperate site
• point planet.<new-domain> to point at this site
• Set up cron job to run generator

Final

Point planet.nottinghack.org.uk at site

Wiki

Has a lot of data, but less traffic than the website. Make sure wiki move it announced publicly We could put a temporary banner on the wiki, similar to wikipedia's fund-raising.

Wiki on nginx guide http://blog.bigdinosaur.org/mediawiki-on-nginx/

First steps

Worth a read http://blog.bigdinosaur.org/mediawiki-on-nginx/

• Set up folder "public_wiki"
• point wiki.<new-domain> at "public_wiki"
• Install mediawiki on new server - version 1.19.2 (to match current)
  • Use wiki-only database
  • Do not set up any additional users
• Set up url rewriting
• Look into APC Cache settings http://www.mediawiki.org/wiki/Manual:$wgMainCacheType
• Install plugins (some of these are part of normal mediawiki install):
  • Interwiki
  • Renameuser
  • Replace Text
  • CSS
  • MagicNumberedHeadings
  • ParserFunctions
  • Poem
  • SyntaxHighlight
  • Widgets
  • ConfirmEdit
  • CategorySortHeaders
  • Vector
  • WikiEditor

Transfer

• point wiki.nottinghack.org.uk at splash page explaining transfer
• Take database dump
• Extract wiki tables
• Transfer media files
• Install database

Final

• Point wiki.nottinghack.org.uk at site
• Put wiki database into backup schedule

Website

First steps

• Update nottinghack.org.uk to latest Wordpress
• Update all plugins

Set up

• Install wordpress on <new-domain> under "public_wiki"
  • Use wordpress-only database
• for easy wordpress upadte and plugin install pre load ssh access details into wp-config
  • seprate ssh key for wp stored under www_secure, need group owner of www-data( check this, should allow for 640 on id_rsa and id_rsa.pub)
  • add key to authorized_keys
  • add lines to wp-config
  • need to check we can get a key with passphrase to work and not have passphrase in config
  • need to double check alternative to this might just be the direct method as php pool runs under NottingHack user
• Install custom theme
• Install wordpress-plugins
  • http://wordpress.org/plugins/w3-total-cache/
  • nginx compatibility http://www.linuxforu.com/2012/02/wordpress-nginx-part-2-domain-vhost-config-migrating-files-fine-tuning-mysql-apc/
• Install custom plugins
• Ensure url-rewriting working correctly

Transfer

We have two options - use the wordpress backup and restore functionality, or dump the database & media files and transfer manually. I suggest we try the backup and restore first, but I seem to remember that this causes a lot of issues with media.

• Put nottinghack.org.uk into maintenance mode

Final

• Point nottinghack.org.uk and www.nottinghack.org.uk at site
• Put wordpress database into backup schedule

Clean up

Other tools, such as dm2tweet will need moving as well. These can't be moved until nottinghack.org.uk is pointing at the new webroot.