Last time, I went through a high-level overview of the project idea. Since then, I've done plenty more brainstorming, and I've been learning a lot more about the protocol.
I said that by the time I wrote this next log, I'd have more of the ATProto nuances worked out, I'd have a draft of a few of the lexicons for this project, and I'd have some more information about how to grab the information from the game. I now have all three of those things, so let's talk about it!
This post gets a lot more into the weeds of specifics about how data is being stored. I've tried my best to include relevant details where I can. Hopefully the less-technical readers can get an idea of what I'm talking about, but it's definitely more dense than the previous log, so feel free to skip around if things are confusing.
Decentralization Decisions
One question at the top of mind since before finishing the first devlog is "who should own a level?" Recall that the entire idea behind the Atmosphere (ATProto) and its philosophy, is that "every user owns their own data". Part of the data required to get an app like this to work is data about the level itself: the name, the difficulty, or whether it’s a part of Strawberry Jam. Since data ownership is a focus of this project, we should spend some time answering who the owner of that data should be.
If we follow the philosophy of decentralization to it's ideal conclusion, the answer to this should be "each user owns the data for levels that they’ve made, then all that data is aggregated together to form a centralized view of all the level data." This principle works great for things like Bluesky, where the creator of each post is the person submitting it. A post cannot exist without somebody to post it, but that’s fine since every post has a creator with an account to submit it.
In Celestat, this doesn't work, because levels in Celeste have already been created, and user statistics rely on it existing. In this scenario, someone would have to claim “creator” of Strawberry Jam and would have to upload the data for Strawberry Jam to their account before anyone could reference it in their own statistics. Rather than hound every creator of every Celeste level to make an Atmosphere account, we should find a different owner for that data.
One way to fix this would be to have an "unclaimed" user that owns any level data for statistic objects that needs to reference a level that doesn't yet exist. This way, you can use any level for statistics even if its creator hasn’t uploaded it to the site.
There's a few issues with this solution. Firstly, if this is the model we pick, we now have to define a whole "claiming level data" process. If someone records statistics on Final Goodbye before Monika makes an account to upload it, the unclaimed user automatically creates Final Goodbye as a level so it can be used. If Monika wishes to claim ownership of the level, migrating it to her account is an extremely messy process. Every user that has ever recorded statistics on Final Goodbye would have to update their statistics record to reference the new record, otherwise all the links break.
Another part of the problem here is that inviting any user to create a level record invites an entirely new set of issues. A user could make their own level that shares the same in-game ID as an existing level, and since Celeste itself has no knowledge of this app, it has no way to distinguish between those other than to ask the user to differentiate between them. There's nothing stopping a malicious user from making hundreds of levels with the same ID, making it needlessly hard to assign the right level record to the “official” level.
All of this is to say, what I’ve come to is users shouldn't be able to own level data itself. This might sound counter-intuitive to the "decentralization" principle running behind this app, but at the end of the day, it’s your submitted statistics that we really care about being owned by you, which will still be the case. In addition, this ensures consistency, as a central level repository would be able to ensure two levels don't have the same in-game ID, and that levels can't be deleted or changed to break existing links. This data will still be held in a PDS, so the data will still be public and can be fetched with the same methods as all the other data.
Lexicon Previews
Now that we have a better idea of where data will be, let's take a preview into what that data will look like!
This is a problem is more difficult to solve than usual, since one of my requirements for this project is the users ability to scope the data collection to what they're comfortable with. Data in just about every app nowadays operates under a "relational" ideology, where different pieces of data have different relationships to each other , such as "user owns post" or "post has likes". When users have full control over their data, and they can choose to opt-out of certain "data types", this model can break. If the answer to “how many likes does this post have” is determined by the number of “like objects” pointing to it, that question can't be answered if the user doesn’t store likes. For this reason, I needed to come up with a self-contained way to store the data.
I've published my first draft of each of the most important lexicons, these are available on PDS.ls, and as a repository where you can open issues if you find them. With that being said, let me do a bit of a deeper dive into the thought process behind the way I’ve structured things.
Stats Slices
I love design patterns where a bunch of related objects can all be combined to one standardized format. Stats Slices are my attempt at that. Here's a simplified structure of a Stats Slice:
type StatsSlice = {
startDate?: Date
endDate?: Date
startTime?: integer
endTime?: integer
startDeaths?: integer
endDeaths?: integer
playedLevel: PlayedLevel
}The slice keeps track of three different data points, the date, which is actual real-life time, the time, which is the in-game timer, and deaths which are deaths in a level. If the only stat being tracked is your sessions "one period of playing the game from game launch to game exit", you can attach a slice to it and answer questions like "How many times did I die during this session" just by taking the difference of the two numbers, or perform actions like “merge these two adjacent sessions together” by taking the lowest start value and the highest end value of each property.
If you are also tracking the locations for each death on a level, this allows you to answer even more questions by linking those deaths to the played level object. The session has a starting and ending boundary of which deaths happened in the session, meaning if you link stats slices to 'clears' and 'rooms' you can get even more info, like "where are all the places I've died in this room" or "how many deaths did this clear take" or "how much total time have I spent in this room" all using the same object. Pretty neat!
(Un)known IDs
Earlier we determined that our level data is going to be effectively centralized, meaning each user has no control over the levels that are available in the app. The question becomes, what happens if you try to submit data on a level that is not known by the app?
Every level in Celeste has an SID, which is what it uses as its identifier for any given map, which takes the form of the maps file path. This is why the naming convention for modded maps is creatorName/modName/levelName, since it’s very unlikely to clash with any other mod the user might have downloaded. Since levels have IDs in-game, we can use this to our advantage here.
The “PlayedLevel” object is the one that represents ‘a users overall statistics playing one specific level’, meaning it should refer to the level record in Celestat if it exists. If it doesn’t exist, it can use the SID as a substitute, along with a temporary name so the user can differentiate them. This way your statistics are able to exist with or without an entry to map it to, and can be transferred easily if it eventually becomes available.
Game Information
Since the last post, I attempted to actually make a Celeste code mod. I've attempted it previously, but it was an attempt at adding a new asset with a custom functionality. Safe to say I was quite unsuccessful. Since then, I’ve done a bit more experimentation with modding, and I’ve made a few important discoveries. Everest, the modding platform for Celeste, provides several event handlers out of the box. I just need to register handlers for LevelEnter, LevelExit, LevelComplete, PlayerDie, GameLoad and GameExit events, and everything else is my own business logic.
I've also played around a little with one of the ATProto libraries for C#, CarpaNet, which Everest refused to load for some reason. I've decided on rolling my own ATProto integration, which for the purposes of this project shouldn't be too extensive. The majority of my time on this project has been trying to get authentication working, which is arguably the hardest part of the protocol, that'll be a significant step forward once its done.
I think that's all of the updates for this time around! I have a lot of questions I'd like answered for next time, like what the merging strategy should be between a local game's statistics and the net statistics should be. I also want to have a prototype of a mod that can actually upload some statistics to a PDS to stress-test any holes that might come from my design. If you're interested in seeing what comes next, check this blog for future updates, and check out my main blog if you want to see other things I'm working on.
See you in the next one!