How to ensure my web app has backward compatibility with importing JSON based user data?
How to ensure my web app has backward compatibility with importing JSON based user data?
I am working on a small web app that stores user data locally using indexedDB which can be imported/exported by making use of JSON files. Since I plan on adding updates to the site, I want to know what best practices I should follow to make sure my app can allow importing of user data from older versions. It could be related to how I should define the properties of my user data object to make it future proof, or any library or tool I could implement that would make this migration process easier.
Do keep these points in mind:
- I am using NextJS to build this application and Dexie to manage indexedDB
- Without going into details, the user data file makes use of heavily nested objects and arrays and most likely won't fit in a cookie or even in the local storage API
- This web app acts as a proof of concept which must only make use of the aforementioned core technologies, regardless of whether more efficient alternatives exist or not.
Two rules of thumb that I've found useful:
- Introduce a version field top-level. If necessary, you can introduce a version 2, but keep the parser for version 1 around.
- You want to err on the side of using objects in place of single values, because you can add fields to an object without breaking backwards compatibility.
You should also have a way to convert from version 1 to version 2. Then if someone loads an old version you can save it back in the new format.
I can’t speak to your framework specifically, but:
I’m assuming this is a REST API. I would suggest versioning the API, like
/api/v1for example.It’s funny, I’m currently dealing with this at work. The old API was just
/apibut on the backend we’ve mapped it toAPI::V1(Ruby). It gets a little pesky backporting things to the older API so it’s good to start with a solid foundation as much as possible.Something else I might suggest: use a good serializer and deserializer. You don’t want to muck up your models with crazy translations for everything: having a middle layer to perform that has been so, so beneficial to us.
Use Open API schema. You can define data models and endpoints or just the models, I do this at work. Then generate your code using openapi-generator.
Do you happen to do this in Ruby on Rails? I don’t know what happened but it seems like Swagger, JSON:API, and the serializers/deserializes are all abandoned.
For personal projects I use GraphQL for everything, I’m not a fan of REST these days. Let me define a schema and let clients screw around with the data. I just won’t waste the time anymore despite the performance impact everyone might cry about.
Honestly not sure about Swagger, I've only ever used swagger-ui to show the API docs on a webpage. OpenAPI as a standard and openapi-generator are not abandoned and quite active. I'll give you an example of how I use it.
I have a FastAPI server in python that defines some endpoints and data models that it can work with, it exports an openapi.json definition. I also have a common schemas library defined with pydantic that also exports an openapi.json (python was chosen to make it easier for other team members to make quick changes). This schemas library is also imported in the FastAPI app, basically only the data models are shared.
I use the FastAPI/openapi.json to generate C++ code in one application (the end user app) using the openapi-generator-cli, serialize/deserialize is handled by the generated code, since the pydantic schema is a dependency of the FastAPI server, both the endpoints and data models get generated. The pydantic/openapi.json is also used by our frontend written in typescript to generate data models only since the frontend doesn't need to call FastAPI directly but it has an option to in the future by generating from FastAPI/openapi.json instead.
This ensures that we're using the same schema across all codebases. When I make changes to the schema, the code gets re-generated and included in the new c++/web app builds. There's multiple ways to go about versioning, but for data only schema I'd just keep it backwards compatible forever (by adding new props as optional field rather than required and slowly deprecating/removing props that are no longer used).
I found this to be more convoluted than just using something like gRPC/Protobuf (which can also be serialized from JSON), I've used it before and it was great. But for other devs that need to change a few lines of python and not having to deal with protobuf compiler, it's a more frictionless solution at the cost of more moving parts and some CICD setup on my side.
Obligatory "JSON APIs are not REST because JSON is not hypermedia".
GraphQL is a mess too as you throw out any ability to reason about query performance and it still requires thick clients with complicated/duplicated business logic.
If you're doing RoR anyway, then go for https://htmx.org/. It's much, much simpler and closer to how the web was originally designed. Highly recommend this book the author wrote on the subject (also provides tutorials walking through building an app): https://hypermedia.systems/book/contents/.