Netstatus 2011 - HADDOW INTERNET

Home, 2024 2023 2022 2021 2020 2019 2018 2017 2016 2015 2014 2013 2012 2010 2009 2008 2007 2006 2005

Haddow Internet Network Status History For 2011



10:30 26-Dec-2011	The backup generator that runs server2 had not had a run in 2 months, so admin started it up, just as it got running there was alot of smoke coming out of the generator, further looking found the windings are in a very bad way. But due to no place open till after the new year, the generator will be out of action. The site has UPS power that will last for about 30min, but after that the server will just shutdown till power is restored. We hope this will not happen. This generator is only 2years old and should not have happen..
16:25 10-Dec-2011	Bigpond And Telstra emails back on line for now, still some services down, there server might go up and down for awhile since there is a lot of people trying to get there emails.
10:00 10-Dec-2011	The internet has gone crasy over this telstra problem, "TELSTRA faces the wrath of the Federal Privacy Commissioner" See: 1. Whirlpool 2. SMH online 3. apples.blogspot 4. Illawarramercury 5. Adelaidenow 6. The Australian IT 7. Cauriermail
09:10 10-Dec-2011	Bigpond And Telstra Service Status pages clame it all started "Friday, 9 December 2011 6:10:00 PM AEDST".
09:00 10-Dec-2011	It has taken 1.5hr on hold for some one to tell me there is a server problem, also the phones at telstra are having problems too, then they tryed to divert me to the right area, just dead line, but that is telstra for you....
07:30 10-Dec-2011	Since about 20:30 last night, any person on a telstra ip address could not get there Telstra emails via pop3, we have been on the phone to telstra but it is taking them over 1hr to answer there phones. We will keep you up to date ASAP.

03:00 16-Nov-2011	Server2 back on line, it was a major outage that took out most of easten Australia, No report from Upstream providers.
00:01 16-Nov-2011	Link to Server2 is down, Looking into the problem.

00:01 01-Oct-2011	Nothing to report for this month.

00:01 01-Sep-2011	Nothing to report for this month.

14:00 11-Aug-2011	One of H.I. Customers are complaining they are not getting any emails, but they clame it started a few days ago, (this is not part of the Outage), the problem is traced back to the wrong domain the sender was trying to send it too, not H.I. problem.
13:31 11-Aug-2011	Update: All back on line, BUT are slow till RAID is finished.
13:20 11-Aug-2011	Update: Servers are coming up one at a time, the server has power to it, we should be up with in 2 hr.
13:00 11-Aug-2011	Update: Upstream provider main links is up, they are starting to repower up the servers, BUT this will take time and fsck needs to be done before the server is up, once the server is up things may be slow for awhile till RAID is finished.
11:00 11-Aug-2011	Update: Server1 still down, this in not just Haddow Internet porblems, over 10,000 people are complaining about there servers in the same center. We are chating with the upstream provider to see why it is taking so long, the RAID system should not take more 10min to check but is taking longer.
10:45 11-Aug-2011	Power returned to server but due to other problems the server is reporting as down
10:20 11-Aug-2011	Update: Power back on link one, UPS back up and 10% charged, Core router1 is backup, rack has power, now we have to wight for the server to rebuild the raid before it come back up. We hope to be back up by 11:00
09:30 11-Aug-2011	Update: One of the A.T.S is not working, the ups has 3 mains links but link 3 had no cable, link 2 was falty a few months ago, that left link one that went down, but bacuse the ATS operates the generator too on generator.
09:00 11-Aug-2011	the data center has lost power via one of there links
02:00 11-Aug-2011	server1 down
19:30 09-Aug-2011	Server1 Load Issues fixed, Problem with the server its self, there was no downtime to the server.
18:45 09-Aug-2011	Load issue on server1, we are looking into why.

15:00 20-Jul-2011	We have upgraded some software on Server1 And Server2 this has help load most websites quicker, and allow for more flexable websites too, there was no down time on both servers duing this upgrade.
15:45 05-Jul-2011	All went well, the network was only down for 10 min.
09:30 05-Jul-2011	We have been told by the up stream provider that there is a problem with one of the core routers. To fix the problem they will be rebooting the router at 0530 UTC time today (15:30 AEST). This will cause a 10 min network outage to Server1. Services affected are: Pri-Mail, Sec-DNS, Pre-Web, Pre-Webmail. Backup services will still be operating and Pre-DNS.
00:01 01-Jul-2011	New Spam filtering has been added to all emails, this service is backed by Junk Email Filter.

13:15 23-Jun-2011	All went well with no problems, Link is back on line.
09:30 23-Jun-2011	We have been told by the up stream provider that there will be some maintenance done on Server1 and its links on 29th June 2011 at 0300UTC Time. This will cause a outage for up to 10min during this time. We will be monitoring for any problems, but don't think any problems will arise. Services affected are: Pri-Mail, Sec-DNS, Pre-Web, Pre-Webmail. Backup services will still be operating and Pre-DNS.

00:01 01-May-2011	Nothing to report for this month.

19:45 27-Apr-2011	Server2 back on line, the server had a few script problems on bootup, but is fixed now too.
19:35 27-Apr-2011	Server2 down, A staff member has pressed the wrong button on the server and it rebooted.
16:05 04-Apr-2011	New core router is up and running, it took awhile to check all vlans.
10:00 04-Apr-2011	We have ben told the core router to Server1 will be upgraded from 01:00utc time and the link may be down for upto 90min, This is it improved IPv6 and fault tolerance support for the router.
05:00 03-Apr-2011	All computers switched over to AEST with no problems.
12:00 01-Apr-2011	Daylight Savings time ends at 3am AEDST on the 3rd of Apr, clocks go back 1hr to 2am AEST.

17:00 19-Mar-2011	Server had a hard robooted and all back online, PHP is up and running, ups software configed and running and all looks to be running like it should be.
12:45 19-Mar-2011	To fix the php problems the server was rebooted, but did not come back online like it should, fixed for now but server will have another reboot for testing later in the day.
22:00 18-Mar-2011	Server2 web server back online but there is still problems, It looks like one of the updates had crashed PHP so Apache would not stay up, Disabled php for now will fix it in the morning.
19:00 18-Mar-2011	Server2 web server is down, we are looking into why.
16:00 17-Mar-2011	New Server up and running now, most files have migrated accross, there was no down time and we now have a stable box to build on if need be.
12:00 15-Mar-2011	New server arived, will be installing it from now.
12:00 08-Mar-2011	We are looking into installing a new server for Server2 with in the next few weeks, there should be no down time migrating systems accross to it.
04:25 01-Mar-2011	Server2 Link back on line, it looks like a problem in Syd on a router.
04:00 01-Mar-2011	Server2 link down, Looking into why.

10:50 27-Feb-2011	Server2 is back online, the modem had a few soft reboots last night to try to fix the problem but it did not work, staff attended the site and power the modem off and back online to fix it, still trying to trace down what caused it.
23:00 26-Feb-2011	Server2 link is going up and down we are tryng to trace down why.
21:20 26-Feb-2011	Server2 is down, we are looking into the problem.
21:20 22-Feb-2011	Server5 back online, Still looking into what happend.
19:50 22-Feb-2011	Server5 is dowm but link is up, it looks like there is a problem with the hole rack (Could be the core router causing the problem) we are looking into the problem and hope to have it back online ASAP.
22:50 01-Feb-2011	Server1 back on line, we are watching to make sure all is stable and hope we don't have a problem again.
22:40 01-Feb-2011	Server1 is requiering a reboot ar 22:45 to fix some mem problems.
16:25 01-Feb-2011	All services back to normal.
16:00 01-Feb-2011	Server1 access is down but the server is still pinging, Looking into it.

12:00 10-Jan-2011	All services back to normal.
08:00 10-Jan-2011	Due to some upgrade problems some backup sites hosted by Aus Server2 may not be viewable, we have traced the problem and are fixing it, Responce to mail and some phone fuctions may be slow for the next few hours till fixed.
04:00 10-Jan-2011	Link back to normal.
23:30 09-Jan-2011	Core Router2 is having a problem and the link to Aus Server2 may go up and down for a few hours till the problem is traced.

All time stated above are in AEST (or AEDST) unless other wise stated. UTC Time Definition, CST Time Definition, AEST Time Definition