PSNC NOC migration Szymon Trocha, PSNC 13.12.2012 Facts and - - PowerPoint PPT Presentation

PSNC NOC migration

Szymon Trocha, PSNC 13.12.2012

Facts and figures

PIONIER

Servers

Air conditioning 24 / 7

POZMAN

Services

Security 3 engineers / 1 operator

Regional

PSU, generators

CCTV Fire detection

Rationales for change

• New services, new responsibilities
• More people, more space
• Room refresh
• Redesign for public viewing
• Equipment detached from operation room
• Better visibility
• Power stability
• Stable network connectivity
• Dispose old workstations
• Accommodate new workstations

Plan…

• Accommodation of new space and room renovation
• New and extended infrastructure

– Fitted for new equipment and space organization

• Replace old monitoring stations with new hardware and software

– Integrate as much as possible

• Deploy new TTS
• Install videowall for large display of network and services status
• Deploy new workstations to control videowall
• Increase NOC team with additional engineers

– Cover new services – Operator 24/7

• Integrate CCTV from multiple locations
• Improve and extend telephone system to fit new room organization and

NOC needs

… results

• Started Dec 2011, still in progress…
• New room
• Dedicated infrastructure
• Videowall
• New workstations (but not main NMS)
• New applications
• Good support from management

Space

• Previously

– ~34 m2 – Old wall painting – Equipment rack in the same room – Not sufficient lights – Insufficient visibility

• Now

– ~63 m2 – Separated room for racks – Painted interior – New floor finish – New lights – Air-conditioning – Operator’s console

Infrastructure

• New power circuits
• Rack Automatic Transfer Switch
• New power sockets, sufficient amount

– Power switch protected for critical workstations – UPS protected

• Cable ducts for signal cables along the main wall
• Electromechanical lock for doors

– Remotely controlled

Hardware and network

• New workstations

– Fujitsu-Siemens

• E.g. 2 CPUs, 64 GB RAM, RAID
• Wireless keyboards

– Windows 7 – Specialized graphic cards – To fully migrate from SUN soon

• New, sufficient network structured cabling

– Minimise long connections and free laying cables

• Used for phone lines too
• Protection and redundancy assured

– Dedicated LAN switch

• Cold standby switch to replace

– VRRP

Software

Nfsen

(NetFlow)

ADVA NMS MRTG

(SNMP)

Request Tracker

iSpy

(CCTV)

FEDERICA map

RIM

(Juniper)

WiFi Poznań

Nagios

Software ctd.

• Also using

– Dude

• Regional network monitoring

– What’s Up Gold

• Commodity Internet

– HP OpenView

• Outdated (sw, hw)
• Changing to new system part of the NOC migration
• Candidates

– HP OpenView  – IBM Netcool 

• Targeting integration of different event sources under one umbrella

– Minimise applications used – Better root cause analysis

Visualization

Visualization ctd.

• Videowall

– 2 x 6 NEC 55” Full HD

• Space allocated for 2 x 8
• Very thin bezel
• LAN control

– Dedicated specialized flat panel CHIEF mounts

• Precise regulation
• NVIDIA Quadro NVS450 graphic cards

– Multi-display graphics solution

• 4 x DisplayPort (FullHD)

– 2 cards in a single workstation (2x), 1 card in a single workstation (2x) – Control panel to control displays and place application windows – Very silent

• No active cooling

– Performance impact when scaling to multiple displays

Still to do

• NMS migration (Jan 2013)

– Change for software, OS, hardware, integration – Failover solution

• Additional furniture to accommodate more persons (Jan 2013)
• New CCTV system (1Q2013)

– Central application – Large screen display

• Telephony

– Additional lines – Mobility

• Integration of new services to monitor (1Q2013)

– Enhanced monitoring – Minimise number of events – New dedicated engineers

• Improve and automate reporting

Lessons learnt

• Decision process is slow

– Even if money exists – Take it into account when planning

• Plan your room infrastructure carefully

– There is always a need for additional socket

• Tools must fit the infrastructure

– Suitable for videowall display – Multi-screen display – Integration

• Make sure NOC team follows changes in tools

– Communicate and train

• Keep you desk clean – there are visiting groups around the corner 

noc.pionier.gov.pl noc.man.poznan.pl