Site Maintenance: AWS Migration (Download Statistics)

EDIT January 19th@9:25am PST
Maintenance and cleanup is complete. Points may seem a little higher than usual, but they will balance out by end of month. Refer here for more information on Reward Points:

EDIT January 17th@12:38pm PST
Previous estimates were inaccurate. Due to unforeseen delays, completion estimate is now January 18th, 2018 at 12:00pm PST 


Greetings Authors!


Later today we will be performing relatively large scale maintenance that affects our download statistics (DLS). The quick and dirty is that we will be moving all of our download statistics to AWS and it will result in a downtime window between 8 and 16 hours starting at approximately 9:30am PST (11:30am CST, 12:30am EST).


After completion, it will take approximately 24 hours for the download statistics to recover and reflect any downloads within that downtime window. You may notice changes to download counts during this time, however, they will not be fully recovered until the job has completed. Author points will not generate on time overnight, they will be retroactively generated tomorrow 1/18 at around 10am PST.


We expect no data loss during this transition, and all download statistics should remain accurate once complete.


What are download statistics?


Download statistics (DLS), is the name we’ve given the resources that work together to log all downloads, from all sources, across the entirety of the CurseForge network. Although the data that we store is relatively simple, we have billions of downloads across hundreds of thousands of projects. Moving DLS to AWS means that we will no longer have to worry about scalability and the hope is that it will allow us to further guarantee accuracy long into the future.


What is AWS?


AWS, or Amazon Web Services, is a collection of services often referred to as “the cloud” that attempts to eliminate issues of scalability, cost, and reliability by pooling many resources together and interconnecting types of resources to provide redundancy and speed across multiple regions.


Why move?


Our servers are not new and they are not unlimited in power or space. The last year has been a little touch and go with different services showing their age, jobs failing, and a variety of other minor technical issues due to various restrictions we have had on our aging hardware.


DLS migration is the first step in several we will be making in the coming weeks to get to new, scalable hardware and services that we can take advantage of as we release new features, fixes, and game support here on CurseForge.


Most importantly, this is a huge step towards increasing the availability of our services in countries that are unable to access our CDN's reliably.


What’s next?


Next week we will be moving the bulk of the CurseForge front and back end. Due to the complexity of DLS, we wanted to get those stats moved over before moving the primary site. We will of course have another blog post detailing the downtime and information regarding that move.


As always, if you guys have any questions or comments, feel free to reach out on Twitter (@CurseForge), or post below.



Your friendly neighborhood MrFlamegoat



Posts Quoted:
Clear All Quotes