.........................

Overview

The Infrastructure Unit operates the following general services across three sites: Informatics Forum, Appleton Tower, JCMB. Each site is set up so that it can operate as autonomously as possible, while at the same time providing redundant services to the other sites.

Network infrastructure and services

Each site's ether switch configuration has been tailored to the particular circumstances. At the time of writing (October 2011) we have 171 switches in the Forum, 43 switches in Appleton Tower, and 2 switches in JCMB.

Each site has a pair of FibreChannel fabrics. In JCMB these are two independent FC switches. The Forum and Appleton Tower are set up as two independent triangles, each having two switches in the Forum and one in Appleton Tower.

The Forum and Appleton Tower each have four network infrastructure machines, as follows:

primary external router
network infrastructure: monitors and configures the site's switches; site secondary nameserver; rfe master for various maps, as required; secondary external router
network services: site nameserver; site VPN endpoint; site WoL host; other site services as required; tertiary external router
external nameserver: carries .inf.ed.ac.uk and all .ed.ac.uk zones, as notified; backup external router; generally an older machine displaced from a previous role

At JCMB we have only two machines: one combining the first three roles above, and the second acting as external nameserver and general standby.

Consoles

Each site has one "console server" machine, acting as a central point for both the site's IPMI and Lantronix-serial consoles. In addition, they act as console server for each other; and in a few cases we have an off-site console server set up for critical machines.

Monitoring

A nagios monitoring service is integrated with our machine configuration system. We have two nagios machines:

A main nagios server, which performs the primary role
A secondary server, which normally just monitors the primary, but which can be swapped into service if the primary fails.

Chat

We operate a chat service, as this is also used by the nagios system to send alerts. The service itself currently runs on the secondary nagios machine.

Authentication services

We use kerberos for authentication. There is one master KDC, currently in the Forum, and one slave KDC at each site. The iFriend master KDC is in the Forum, with a slave in Appleton Tower, and an additional slave at KB (this one is non-operational and purely to provide a backup away from the central area)

Although we don't make a lot of use of kx509 at the moment, we still run a kx509 service. There are currently two KCAs, both on VMs.

Most web authentication now uses cosign. We currently have cosign servers in the Forum and Appleton Tower.

Hosts and services requiring a locally-signed X.509 certificate obtain this using the sixkts service. As this is not a high-availability requirement, we currently have one sixkts server on a VM.

Directory services

The OpenLDAP master is currently in the Forum, and there are site slaves in the Forum, Appleton Tower and JCMB. In addition, all DICE machines also run a full slave configuration.

Account management (prometheus)

The prometheus system runs on a machine in the Forum. This would not be an easy service to replicate, but as high availability is not a requirement we could move the service to another machine should the main one be unavailable. This could be either the development server or the JCMB OpenLDAP server, as decided at the time.

Infrastructure Unit Kit Lists

Linux servers

(Sorted by date and then hostname.)

Name	Type	Location	Role	S/N	P/O & date; warranty	UPS (if non-building)	Comments
berlin	PE850	AT server room	cosign, ifriend KDC slave	HKD222J	A634457 2006-01-18
osprey	PE850	Forum server room	cosign, ifriend KDC master	7KD222J	A634457 2006-01-18
wallace	SC1425	JCMB server room	extDNS	C7RY12J	A634457 2006-01-18	5kVA CS0543110262	Formerly "mustang"

darwin	SC1425	AT server room	extDNS, jabber logs	GJFHB2J	A636553 2006-06-22		Formerly "levine"
linnaeus	SC1425	Forum server room	extDNS	3KFHB2J	A636553 2006-06-22		Formerly "karajan"

franklin	PE2950	Forum server room	LDAP master	BJT4V2J	IKB0053 2007-03-26

crystal	GX745	Forum 5A closet	NTP S1, DHCP	428903J	ikb0153 2007-06-22		"Real" serial port required

curlew	PE860	AT server room	Nagios secondary	HL7YY2J	IKB0115 2007-06-05

ancerl	R200	AT server room	AT netServ, extNTP	H660D4J	inf0488 2009-06-30
blackwell	R610	AT server room	LDAP site-slave	9GD2D4J	inf0488 2009-06-30
fenrir	R200	Forum server room	KDC, AFSDB	J660D4J	inf0488 2009-06-30
hickox	R200	Forum server room	Forum netServ, extNTP	C560D4J	inf0488 2009-06-30	3kVA JS0617023553
mckinley	R610	Forum server room	LDAP site-slave	BGD2D4J	inf0488 2009-06-30

hati	R210	JCMB server room	KDC, AFSDB	GJ6TJ4J	inf0622 2009-11-02
skoll	R210	AT server room	KDC, AFSDB	FJ6TJ4J	inf0622 2009-11-02

verte	R310	AT server room	AT consoles, DHCP	HYQNR4J	inf0879 2010-05-21

WARRANTY EXPIRED ABOVE HERE, based on a 3-year standard warranty

otaka	DL180	AT server room	AT netInf, DHCP	CZ30291JCP	inf0953 2010-07-09; 4Y	1kVA AS0614310419
reeves	DL180	Forum server room	Prometheus (former master)	CZ30301P1K	inf0953 2010-07-09; 4Y

abbado	DL180	Forum server room	Forum netInf, DHCP	CZ3115CJ62	inf1175 2011-03-31; 5Y	3kVA YS0351120138 3kVA XL QS0348111013

cockerel	DL180	Forum server room	Nagios master	CZ3121H23L	uoe26808 2011-05-18; 5Y
hall	DL180	JCMB server room	LDAP site-slave/prometheus DR	CZ3121H23F	uoe26808 2011-05-18; 5Y
tycho	DL180	Forum server room	loghost	CZ3121H23H	uoe26808 2011-05-18; 5Y	3kVA JS0510014805

slatkin	R310	JCMB server room	KB netServ, consoles, ifriend KDC slave (unused)	4WL3C5J	inf1642 2012-05-10; 5Y	3kVA JS0510018437

elder	R320	JCMB server room	KB extRt, netInf, DHCP	BB3YC5J	inf1748 2012-06-18; 5Y	3kVA JS0511022966
norrington	R320	AT server room	AT extRt	CB3YC5J	inf1748 2012-06-18; 5Y	1400VA XL QS0322110541 1400VA XL QS0322210008

bevan	R320	Forum server room	KDC master	B15CD5J	inf1771 2012-06-22; 5Y

blatiere	R320	Forum server room	Forum consoles master, DHCP	J9MM9X1	inf2540 2013-03-28
rattle	R320	Forum server room	Forum extRt	G1KL9X1	inf2540 2013-03-28	3kVA XL QS0348111013 3kVA YS0351120138

dammers	VM	piccadilly (KB)	sixkts, kca
buchanan	VM	jubilee (IF)	kca, misc
vandellas	VM	metropolitan (IF)	Prometheus master

Warranty is assumed to be three years, unless otherwise stated. Machines in the Forum and AT server rooms are covered by the inbuilt UPSes, and are shown with a blank in the column unless they have some additional provision. Machines in the JCMB server room may be powered by one of the "rack" UPSes, and in this case are shown with a blank in the column unless they have some additional provision.

FibreChannel Switches

(Sorted by date and then hostname. All warranties have now expired.)

Name	Type	Location	Role	S/N	P/O & date
fc00	SANbox 5600	IF-B.02		0835C00819	ikb0626 2008-09-05
fc01	SANbox 5600	IF-B.02		0834C00021	ikb0626 2008-09-05
fc0a	SANbox 5600	AT server room		0834C00108	ikb0626 2008-09-05
fc10	SANbox 5600	IF-B.02		0834C00018	ikb0626 2008-09-05
fc11	SANbox 5600	IF-B.02		0834C00113	ikb0626 2008-09-05
fc1a	SANbox 5600	AT server room		0834C00026	ikb0626 2008-09-05
fchs	SANbox 5600	IF-B.02	hot spare	0835C01123?	ikb0626 2008-09-05

fc0	SANbox 5800	JCMB		1005F00659	inf0778 2010-03-09
fc1	SANbox 5800	JCMB		1005F00525	inf0778 2010-03-09

UPSes

(Sorted by location and role, more or less...)

Type	Role	Location	S/N	P/O & date	Rating	Battery
SMART-UPS 1400 RM XL	AT server room	comms cabinet	QS0322110541	a602939 2003-06-06	1400VA	2011-06
SMART-UPS 1400 RM XL	AT server room	comms cabinet	QS0322210008	a602939 2003-06-06	1400VA	2011-06
Smart-UPS 1000 RM	AT server room	rack 2	AS0614310419	??	1kVA
Smart-UPS 3000 RM	AT3	comms cabinet	JS0510018446	a631778 2005-07-11	3kVA	2011-07
Smart-UPS 3000 RM	AT4	comms cabinet	JS0617023554	a637388 2006-06-27	3kVA	2011-11
Smart-UPS 3000 RM	AT5	comms cabinet	JS0510018447	a631778 2005-07-11	3kVA
Smart-UPS 3000 RM	AT6	comms cabinet	JS0511022967	a631778 2005-07-11	3kVA	2012-01
Smart-UPS 3000 RM	AT7	comms cabinet	JS0714011678	ikb0141 2007-06-19	3kVA
Smart-UPS 3000 RM	AT8	comms cabinet	JS0714011688	ikb0141 2007-06-19	3kVA	2013-02
SMART-UPS 3000 RM XL	Forum core0 ("core")	comms racks	QS0348111013	a614789 2004-02-04	3kVA	2013-01
Smart-UPS 3000 RM	Forum core1 ("netInf")	comms racks	YS0351120138	a614789 2004-02-04	3kVA	2012-05
Smart-UPS 3000 RM	Forum core2 ("netServ")	comms racks	JS0617023553	a637388 2006-06-27	3kVA	2011-07?
Smart-UPS 3000 RM	Forum loghost	comms racks	JS0510014805	a631753 2005-06-15	3kVA
Smart-UPS 1500	Test & development	IF-1.09	YS0315121217	a609907 2003-08-14	1500VA	2013-03
Smart-UPS 3000 RM	JCMB server room	Rack 0	JS0511022966	a631778 2005-07-11	3kVA
Smart-UPS 3000 RM	JCMB server room	Rack 1	JS0510018437	a631778 2005-07-11	3kVA
SMART-UPS 5000 RM DL4	JCMB server room	Rack 2	CS0543110262	??	5kVA

IUkit.html,v 1.243 2013/05/23 06:14:03 gdmr Exp

: Units : Infrastructure : Documentation

Please contact us with any comments or corrections.
Unless explicitly stated otherwise, all material is copyright The University of Edinburgh