Alert weather watchers (do you guys ever take a day off??!!!??) noted that the site went a little sideways yesterday. The main thermometer thought the high and low for the day was 32°F, the winds and other basic parameters were not right, and some of the supplementary pages failed to load or generated a pageful of errors. Hopefully, it’s working better today.
A junior web operator who shall remain nameless attempted to check the web sites’ configuration files into Git and instead loaded in the entire site, including dynamically updated data files and cached files. This means Git will pretty much always report the repository is out of date, and make cloning amd redeployment a serious hazard. In fixing the issue, the problem compounded: it turns out the “git rm” command does not remove the file from git as one might suspect, but instead OBLITERATES THE WORKING FOLDER OF PRODUCTION DATA and marks the file retained in (for historical purposes), but no longer active in the repository. Removing live data and cached files while the website is running had predictably disastrous results. Management has been notified and stern lectures delivered about diddling with the live production site without making three backups while clicking your heels together.
The management wishes to apologize to anyone who just wanted to know what the weather was outside. Yesterday, you would have had to go out IRL and look for yourselves; today, you can check on your phone again.
It's Another Yankee Forecast Today