The independent student news publication at Kansas State University

Kansas State Collegian

Kansas State Collegian

The independent student news publication at Kansas State University

Kansas State Collegian

Letter from the Editor: Collegian website outage report, Dec. 28, 2023

What happened to our server, why our website was down for the day, why stories were temporarily gone and our plan moving forward

The Collegian website went down for most of the day Dec. 28, 2023, and when it was available again, stories from two weeks prior were erased. While a campus-wide cybersecurity incident is happening right now and is affecting our website at the time of publication, we still want to address what happened last month for transparency.

At 10:15 a.m. Dec. 28, 2023, the website went down because of an internal issue. We run our websites on virtual machines on a bigger server, and we take snapshots of these virtual machines every couple weeks to have backups in case of a major incident. However, our snapshots started to take up so much space they filled up the server’s memory, preventing our virtual machines from running properly and causing our websites to crash.

This is not the first time this has happened, and usually we delete old snapshots to free up storage for the virtual machines to work again. We did apply this, but another problem with the machine arose.

When attempting to turn our virtual machines back on, they did not reboot properly. Every time we turned them on, they would repeatedly shut themselves off and back on again until they eventually stopped.

We have a solution for this situation as well, but in our haste to get the server back up, we pulled our last resort option of reverting to a previous backup from Dec. 13, 2023. This solution deleted all stories between the 13th and the 28th, but on the bright side, our website was back up and running just in time for our Pop-Tarts Bowl post-game coverage.

Within the next two days, we had all our stories reposted with their original dates and almost all sponsored content restored. We have backup documents for all our stories, pictures and content, and with the help of the Wayback Machine, the lost stories were republished to their original states. Except for an added editor’s note and reset view counts, the stories are exact replicas of the originals — including the urls, so social posts and shared links still work.

We apologize for the inconveniences this may have caused, and we are taking steps to ensure this doesn’t happen in the future. We will not pull the last resort option until after affected Collegian Media Group leaders mutually agree to it and only when absolutely necessary. We will also take more preemptive measures to prevent this, including doing a better job of watching our server’s memory.

Thank you for your continued support of the Collegian. If you have any questions, please direct them to IT manager Garrett Brown at [email protected] or to editor-in-chief Carter Schaffer at [email protected]. We will continue to provide transparency when these incidents occur, including when the Jan. 14 campus-wide cybersecurity incident is resolved.

Leave a Comment
About the Contributor
Carter Schaffer
Carter Schaffer, editor-in-chief
EIC for 2023-24. Previously EIC for 2022-23 and writer and photographer for spring 2022.
More to Discover
Donate to Kansas State Collegian

Your donation will support the student journalists of Kansas State University. Your contribution will allow us to purchase equipment and cover our annual website hosting costs.

Donate to Kansas State Collegian

Comments (0)

All Kansas State Collegian Picks Reader Picks Sort: Newest

Your email address will not be published. Required fields are marked *