Network Issues
| Resolved Issue | Priority | Time Frame | Affecting | Status | ||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Hypervisor Failure | Critical | 04/12/2020 9:49:00 AM - On Going | VpsHost | Resolved | ||||||||||||||||||||||||||||||
Detailed InformationDescription: We are currently experiencing an issue with one of our hypervisors in the Grafton data centre. Time Frame: 4/12/2020 09:49 - Ongoing Impact: Some clients with VPS servers are experiencing issues connecting to their servers as one of our hypervisors stopped responding. Our engineers are working on fixing the problem and restoring services. Contact: If you have any issues or concerns about this, or have any queries about the network downtime, please contact us as per our Contact Us page (https://vpscity.co.nz/contact-us) Important Updates (Latest appear first)Friday, December 4, 2020 at 10:09:53 AM All services have been restored. |
||||||||||||||||||||||||||||||||||
| Unplanned Outage / Multiple Failures | Critical | 24/11/2020 8:48:00 PM - 25/11/2020 1:26:00 PM | Some Cloud Servers in the GRF cluster | Resolved | ||||||||||||||||||||||||||||||
Detailed InformationDescription: We are currently experiencing an unplanned outage in our Grafton data centre. Time Frame: 24/11/2020 20:41 - Ongoing (NZDT) Impact: Our engineers are working on fixing the problem and restoring services. Contact: If you have any issues or concerns about this, or have any queries about the network downtime, please contact us as per our Contact Us page (https://vpscity.co.nz/contact-us) Important Updates (Latest appear first)Tuesday, November 24, 2020 at 11:27:15 PM All servers are back online and faulty SFP has been replaced. Tuesday, November 24, 2020 at 10:31:52 PM We are still booting servers on the GRF cluster and it is taking a bit of time so please bare with us if your server is down it WILL come up as soon as we are able to boot it. Tuesday, November 24, 2020 at 09:15:41 PM We have found a faulty Fibre Channel SFP and are switching over to backup. Some servers require rebooting and we are in the process of doing this. |
||||||||||||||||||||||||||||||||||
| Network Issue | Critical | 23/11/2020 1:40:00 PM - On Going | Network | Resolved | ||||||||||||||||||||||||||||||
Detailed InformationDescription: We are currently experiencing intermittent Network Issues affecting all our clients. Time Frame: 23/11/2020 13:40 - Present Impact: All our clients are experiencing intermittent network issues. Our engineers are actively investigating to resolve the issues. Other: We apologise for the inconvenience caused. Important Updates (Latest appear first)Monday, November 23, 2020 at 02:42:54 PM This issue has been resolved and network operations are back to normal. |
||||||||||||||||||||||||||||||||||
| Planned Outage / Scheduled Maintenance | Critical | 04/11/2020 9:00:00 PM - 05/11/2020 5:19:43 AM | Dedicated / Colo Servers in Grafton DC | Resolved | ||||||||||||||||||||||||||||||
Detailed Information
|
||||||||||||||||||||||||||||||||||
| Planned Outage / Scheduled Maintenance | Critical | 01/11/2020 9:00:00 PM - 03/11/2020 2:59:14 AM | Dedicated / Colo Servers in Grafton DC | Resolved | ||||||||||||||||||||||||||||||
Detailed Information
Important Updates (Latest appear first)Monday, November 2, 2020 at 03:48:31 AM All upgrades for this outage window have been completed successfully without incident. |
||||||||||||||||||||||||||||||||||
| Unplanned Outage / CloudHost Cluster Failure | Critical | 28/10/2020 10:07:00 PM - 01/11/2020 1:06:49 PM | Hypervisor cluster GRF | Resolved | ||||||||||||||||||||||||||||||
Detailed InformationDescription: We are currently experiencing an issue with one of our cloud host Nodes in our Grafton data centre. Time Frame: 28/10/2020 21:41 - 29/10/2020 00:57 (NZDT) Impact: Some clients with cloud servers are experiencing issues connecting to their servers as one of our nodes has crashed. Our engineers are working on fixing the problem and restoring services. Contact: If you have any issues or concerns about this, or have any queries about the network downtime, please contact us as per our Contact Us page (https://vpscity.co.nz/contact-us) Important Updates (Latest appear first)Thursday, October 29, 2020 at 01:15:30 AM All servers are up and running optimal now. Thursday, October 29, 2020 at 12:51:55 AM We still have one hypervisor still rebooting and ETA is about 10 min. Thursday, October 29, 2020 at 12:48:05 AM The problem affected one of our clusters comprising of 9 hypervisors. We have resolved the issue and are just in the process of checking all servers to make sure they are at optimal performance. |
||||||||||||||||||||||||||||||||||
| Node Issue Grafton | High | 16/10/2020 12:02:00 PM - 18/10/2020 8:07:14 AM | vpshost-hv2.grf | Resolved | ||||||||||||||||||||||||||||||
Detailed InformationDescription: We are currently experiencing an issue with one of our VPS Host Nodes in Grafton data centre - vpshost-hv2.grf Time Frame: 16/10/2020 12:02 - Present Impact: We are aware of an issue with one of our VPS nodes that will cause service disruptions on certain servers. Our engineers are actively working on the problem and restoring services. Contact: If you have any issues or concerns about this, or have any queries about the network downtime, please contact us as per our Contact Us page (https://vpscity.co.nz/contact-us) |
||||||||||||||||||||||||||||||||||
| Routing Problems | Critical | 21/09/2020 9:15:00 AM - 21/09/2020 10:36:39 PM | All Services in NZ | Resolved | ||||||||||||||||||||||||||||||
Detailed InformationDescription: We are experiencing a BGP routing loop causing routing problems on our network. Time Frame: 21/09/2020 09:15 - 21/09/2020 09:35 Impact: All services on the VpsCity network are experencing outages. We are aware of this and our engineers are working on tracking down the cause and resolving it. Other: This is affecting all clients on our New Zealand network. Important Updates (Latest appear first)Monday, September 21, 2020 at 09:49:02 AM We have cleared the BGP routing loop on our routers which has restored connectivity to all services. We apologize for the inconvenience caused during this time. |
||||||||||||||||||||||||||||||||||
| NZ Core Router failure | High | 18/09/2020 1:00:00 PM - 21/09/2020 9:40:49 AM | Core Router Failure Affecting All Services in NZ | Resolved | ||||||||||||||||||||||||||||||
Detailed InformationDescription: We are currently experiencing failure on one of our core routers in our 220 Queen St datacenter. This is affecting all services in New Zealand. Time Frame: 18/09/2020 13:00PM - 18/09/2020 16:20PM Impact: All clients services in NZ are currently experiencing downtime due to a core router failure. Our engineers onsite and actively investigating the problem, all services should be up shortly. Other: We apologise for the inconvenience caused. Contact: If you have any issues or concerns about this, or have any queries about the network downtime, please contact us as per our Contact Us page (https://vpscity.co.nz/contact-us) Important Updates (Latest appear first)Sunday, September 20, 2020 at 04:00:22 AM The faulty router had a firmware bug that caused the problems and we have now updated it and returned it back into service. Friday, September 18, 2020 at 05:13:59 PM We have failed over to our backup router until we replace/repair the faulty router. There will be no further downtime. Friday, September 18, 2020 at 04:20:00 PM Our backup routers have come online and all services are back online. |
||||||||||||||||||||||||||||||||||
| Ongoing DDoS attacks | High | 01/09/2020 8:50:00 AM - 08/09/2020 2:27:52 PM | N/A | Resolved | ||||||||||||||||||||||||||||||
Detailed InformationDescription: We are currently experiencing intermittent Network Issues affecting all our clients. Time Frame: 01/09/2020 08:50 - Current Impact: Some clients are experiencing intermittent national connectivity to and from their servers. Our engineers are actively working on this to regain full connectivity to affected clients. Please check back here for updates. Contact: If you have any issues or concerns about this, or have any queries about the network downtime, please contact us as per our Contact Us page (https://vpscity.co.nz/contact-us) Important Updates (Latest appear first)Thursday, September 3, 2020 at 02:31:08 PM We are still seeing several DDoS attacks daily and are mitigating them. This should not be affecting services and we are just informing clients of the ongoing situation. Tuesday, September 1, 2020 at 06:16:53 PM We are seeing several large DDoS attacks on our network that we are mitigating, please bare with us.... Tuesday, September 1, 2020 at 10:40:55 AM The network is now stable while engineers are working to completely resolve the problem and restore optimal performance. |
||||||||||||||||||||||||||||||||||