Operational Meeting

Apologies for absence

Alastair, Alison.

minutes of the last meeting.

These were accepted.

1. SL5 Upgrades

   All SL5 server upgrades to be be done by end of March. This is a tight deadline but we don't want this to overrun into preparing the Forum. Priority should be given to external facing servers. Ian has a web page for coordinating the upgrades.

3. Innovation meeting on personal/mobility computing

   Ken noted that even though this meeting will happen Feb/Mar nothing will be done or started until Autumn due to other constraints on our time.

Reports from units.

• Infrastructure.
  1. SL5

     George noted that he would make this flip straight after the meeting and would mail out when done in case anyone spotted any problems.

• Managed Platform.
  1. GDM/X problems

     Stephen commented on the GDM problems noticed during the lab exam. This occurs almost exclusively on older hardware (GX270s and HPs) and results in the graphics display being locked (black) while keyboard and mouse are still responsive (it is possible to switch to the console and type blind for example, to reboot do Ctrl+Alt+F1 and then Ctrl+Alt+Del). Not clear whether its a BIOS or driver problem. The only current workaround is to reboot. Since this only occurs with the lab exam and the older machines will drop out of service soon we don't consider it worth what might be significant effort to fix.

  2. Old code archival

     If old code is found to still be needed it can easily be copied back by logging onto the cvs server and mv'ing the appropriate cvs repository from dice_archive back to dice. The LCFG/DICE separation will need to be reviewed in the future, particularly as more people outside of Informatics make use of lcfg components (it is likely most components could be lcfg rather than dice). The Ed level is likely to move to a central repository in IS.

  3. SL5.1

     SL5.1 will go into the develop release next Monday (28th Jan) to get reasonable exposure before going into testing. You can add dice/options/test_updates.h to get it now.

  4. Moving wires

     This probably only applies to moving machines across our current wires and is not ready for the Forum moves yet as it only supports /24 subnets.

• Research and Teaching.
  1. Lab Exams

     Following experience from the last lab exam we will be modifying the setup to disable USB keys and CDs. We will also disable crontabs - largely because of the log message distraction they cause. There were two or three machines that had very strange NFS problems where the server would not communicate correctly with them even after rebooting. Tim will pass the logs over to the Services Unit for review.

  2. May Demos

     Neil noted that there will be a large Schools visit in May where approximately 300 schoolchildren will be using our machines (in AT) for workshops and demonstrations as part of the outreach program. This will likely mean converting a lot of lab machines into conference kiosks or similar. George will check if there is any network/infrastructure related work going on in AT then that may affect this.

• Services.
  2. Room Booking Software

     This is now in use by the ITO. All data was transferred automatically from the old system. Forum bookings can only be made by the ITO.

  3. More disk failures

     Query as to whether the disks failed because of higher temperatures than normal in the JCMB machine room or whether it was just that the disks would have had the same age. However, within the bounds of probability that our replacement policy was based on.

• User Support.
  1. RT Statistics Report

     Ken noted that these statistics covered the period from the last meeting to the meeting today rather than the normal two weeks which is why the resolution rates are higher.

  4. Removing Old Accounts

     Ken noted the onerous work done by Lindsey in clearing 1000 old accounts.

The following topics were discussed:

• Appleton Tower Power Shutdown - Lessons Learned
  • We overlooked the fact that the Bush and Darwin are part of AT (virtually) so the move of AFS directories to the Bush was never going to work as planned.
  • Because the external routers are in AT the AFS DB servers were visible internally to the School but not externally.
  • Suggestions for alleviating the above were to do short (5min) simulated shutdowns and to put plans for maintaining services to a wider audience to pick holes in them.
  • Multiple IP addresses for www.inf worked during testing but did not work completely on the day (possibly due to browsers/hosts cacheing the previously used IP address and because of differences in how hosts handle 'connection refused' and 'timeout' errors).
  • This was the first time we have tried to maintain services (home directories and web) across an outage like this.
  • We don't spend enough time/resource on providing a fully redundant and resilient 24/7 service - mainly because it is exorbitantly expensive to do so.
  • May be worth playing with turning off random services as we deploy in the Forum to check for points of failure in our redundancy setup.
  • All announcements of this kind of outage should go to 'sys-announce' and not selected sub groups.
  • We still cannot do failover routing because IS run OSPF and we run RIP - however this is going to have to be fixed for when we move into the Forum.
• Porting of web services to SL5 - use of apacheconf component

  The 'apacheconf' component is ported to SL5 and is okay for people to use when upgrading servers to SL5. Neil believes the 'apache' component should work fine under SL5 but it has not been tested. For simple web services converting to use 'apacheconf' is the recommended approach and should only take half a day or so to make the switch. If a service is using 'apacheconf' then any changes made to the cosign infrastructure will be automatically applied to that service - such changes need to be made manually when using the 'apache' component.

AOCB

Simon drew our attention to 'CSRF' (Cross Site Redirection Fraud) so as anyone running web applications should check whether it affects them. Generally this affects users where they leave themselves signed on with one web service (such as googlemail) and then continue to browse the web - their authentication is retained in the browser and malicious web sites can make use of it, so best to always sign off before carrying on with browsing. Our sites are more susceptible because people are effectively always signed on with Cosign authentication. The fix is to embed hidden random codes in forms that can be checked by the server on a GET/POST operation and that a third party would not have access to or could predict.

The next meeting will be on February 13th in the Buccleuch Place Seminar Room and chaired by George Ross.