ceres/model/database.go, branch step-re-orderingRecipe server for your favorite disheshttps://cgit.xengineering.eu/ceres/atom/model/database.go?h=step-re-ordering2024-10-21T19:37:54Zmodel: Rename to db.go2024-10-21T19:37:54Zxengineeringme@xengineering.eu2024-10-21T19:37:54Zurn:sha1:f917bc9e1973bfc1c6ab56bf1362295f89d687c3
The old name database.go did not match the type name DB.
model: Refactor public API of DB2024-10-21T19:34:31Zxengineeringme@xengineering.eu2024-10-21T19:30:12Zurn:sha1:72601b87ef040a3c6882368ac85c12c1ae705cd2
This commit makes not externally needed methods private and adds error
return values since something like log.Fatal() should be called outside
this package since it is control-flow-related.
model: Save int-based schema version in DB2024-10-21T19:34:26Zxengineeringme@xengineering.eu2024-10-15T19:23:10Zurn:sha1:118e1c69057e4e7b6ab3e730d5628b4822ed4c4d
Ceres v0.4.0 used the `git describe` output as database schema and
enforced exactly matching versions between the database and the
executable.
This turned out to be not flexible enough. It is way easier to version
the database separately with a simple integer and require the same
database schema version integer between the application and the
database.
This commit implements this new approach.
model: Handle schema versions internally by integers2024-10-15T18:25:38Zxengineeringme@xengineering.eu2024-10-13T18:34:29Zurn:sha1:1904f50084660ce587a67d49939fd0734fd8a582
This references the database schemas known from the past with integer
values internally. 0 is used for an empty database and 1 for what was
used in ceres version 0.4.0.
The old approach to write the `git describe` output into the database
worked well for released versions but was problematic during
development. A numerical and separate database schema versioning is
easier to handle.
model: Simplify DB method signatures2024-10-13T18:21:00Zxengineeringme@xengineering.eu2024-10-13T18:21:00Zurn:sha1:5ca945112b5cd7c92eb3b9da81597fcba5256b56model: Replace global db variable by custom type2024-10-13T17:55:38Zxengineeringme@xengineering.eu2024-10-13T17:52:28Zurn:sha1:473052ed8f2c83052ed5b47a7f4cec68ac2621a6
Reducing global variables makes it easier to understand functions
independently of the rest of the code.
Adding the new model.DB type as a custom variant of the sql.DB type
makes it possible to write methods for the database which makes the code
way more readable.
Introduce xengineering.eu/ceres/model/migrations2024-10-13T16:54:18Zxengineeringme@xengineering.eu2024-10-13T12:08:26Zurn:sha1:ed19b82335345833c5b8f5446237d559a3657a35
This new package is only for database migrations. All data for Ceres
should be stored inside the SQLite3 database. Thus migrations can always
be executed with functions with the following signature.
func(tx *sql.Tx) error
Those migration functions should be stored inside the new package.
Bigger SQL code can be stored in *.sql files for better syntax
highlighting. This code is embedded into the final Go executable by
using the embed package.
Inject examples only with new --example flag2024-05-09T20:37:03Zxengineeringme@xengineering.eu2024-05-09T20:33:42Zurn:sha1:0ac3cc76b4b6c824c4b3f7a357d40b487984abfb
The default use case should be to not inject example recipes.
model: Rename version to execVersion2024-05-09T20:12:54Zxengineeringme@xengineering.eu2024-05-09T20:12:54Zurn:sha1:f7fca11c47224ff28b10a7f41fb76ce0404a0236
This makes the code easier to understand because there is an executable
version and a database version handled inside that file.
model: Require same version for executable and DB2024-05-09T20:04:06Zxengineeringme@xengineering.eu2024-05-09T20:04:06Zurn:sha1:0e8a199d6144fcfab05eb0c8a793073bc53783be
Currently only an empty database and an existing database with the same
version are supported.
Support for migrations based on semantic versioning will be added in
future versions of Ceres.
model: Migrate only in empty databases2024-05-09T19:59:01Zxengineeringme@xengineering.eu2024-05-09T19:59:01Zurn:sha1:e5d57f2c4a9f01ac7a2b11c04b932311bd240611model: Initial database version injection2024-05-09T19:48:56Zxengineeringme@xengineering.eu2024-05-09T19:48:56Zurn:sha1:ca96df29085f9ca6567cd474d0920e48c6410b42
If the database was empty on startup a metadata table with a key and
value row is created.
In addition the Ceres executable version is inserted as value under the
key 'version'.
This allows to detect on not-empty databases which Ceres version was
used before which is the starting point to implement migrations.
model: Detect if database is empty2024-05-09T19:31:19Zxengineeringme@xengineering.eu2024-05-09T19:31:19Zurn:sha1:ca19c73b305932a4ef41d31787eabb25f9417a05
An empty database requires to add the metadata table with the version
entry to make migrations possible. Thus this detection will be required.
model: Wrap migration completely in transaction2024-05-09T19:10:50Zxengineeringme@xengineering.eu2024-05-09T19:10:15Zurn:sha1:2db08f564ddb38e80c79c9338047f4e8f3a977e8
This makes it more clear that the full migration will be rolled back on
errors.
model: Use defer for tx.Rollback()2024-05-09T09:13:46Zxengineeringme@xengineering.eu2024-05-09T09:13:46Zurn:sha1:fd7ee91110cf1de780f3c46822dd69385b3b6318
A committed transaction cannot be rolled back. Using defer to roll back
guarantees that the transaction is always rolled back if not the commit
in the last line of model.Transaction was excuted.
[1]: https://go.dev/doc/database/execute-transactions
Restructure database-related functions2024-05-09T08:02:00Zxengineeringme@xengineering.eu2024-05-09T08:02:00Zurn:sha1:a2038b0ea35d1466c84e6e04a2e4597fc038815aIntroduce model.Transaction()2024-05-08T20:26:00Zxengineeringme@xengineering.eu2024-05-08T19:53:13Zurn:sha1:0ba1a7661a81200db98e40149eef1e39fd22f407
It is a very common pattern that some function needs to access the
database and wants to wrap all the actions into one transaction.
The advantage of a transaction is that it is ACID:
- atomic
- consistent
- isolated
- durable
In Go it is required to request a new transaction, execute functionality
on it and handle rollback or commit of this transaction based on the
success of the operation.
All this and the error handling can be written down in the
model.Transaction() function exactly once. The full signature of it is:
func Transaction(f func(*sql.Tx) error) error
It requires a function or closure passed as argument which takes the
transaction (*sql.Tx) and returns an error which might be nil.
This is very generic. It is applied to:
- injecting test data
- database migrations
- data read requests
- data write requests
model: Init database with database path2024-05-07T19:25:19Zxengineeringme@xengineering.eu2024-05-07T19:25:19Zurn:sha1:75c1270c86ac17d056161714ac32a57113696d21Use default storage path instead of temp dir2024-05-01T11:19:05Zxengineeringme@xengineering.eu2024-05-01T11:19:05Zurn:sha1:4cc0677b79edeed05ead23def152ac45f35f556dmodel: Always pass *sql.Tx to CRUD methods2024-04-06T11:12:17Zxengineeringme@xengineering.eu2024-04-06T11:12:17Zurn:sha1:f65b11a5b3011f370df5b4d32239225f3708ecd5
When nesting objects like steps into other objects like recipes it is
required to pass a *sql.Tx value to the CRUD methods of the inner
object to be able to roll back the whole transaction.
The top level object used to be responsible for the creation of this
*sql.Tx inside its CRUD methods.
This is now moved to the caller of the CRUD methods (here the HTTP
handler function). The advantage is that all CRUD methods now accept a
*sql.Tx as only argument which makes those methods more consistent.
model: Add recipe steps2024-03-24T09:10:54Zxengineeringme@xengineering.eu2024-03-10T09:59:23Zurn:sha1:ddc6e8e1fb8a4b1dfe8421b12e40b15162ae75f1
This provides the infrastructure to create views and HTTP handlers to
provide recipe steps.
model: Crash on failed test recipe injection2024-03-24T09:10:54Zxengineeringme@xengineering.eu2024-03-12T17:25:48Zurn:sha1:0520cf604c4dc3279d76e1dd4b49c275dd54611e
This error used to be silent. Since it is just about test recipes and
thus a debugging environment it is best to directly give up and log the
error.
model: Create test data with Go instead of SQL2024-03-04T19:33:58Zxengineeringme@xengineering.eu2024-03-03T19:06:40Zurn:sha1:b62c686702622b747ab082b7b12b3e611e1d7e1f
This allows to formulate the test data with an object-based model which
is easier than writing it down in a relational model.
model: Inject test data into database2023-12-27T12:05:05Zxengineeringme@xengineering.eu2023-12-27T12:04:26Zurn:sha1:8ce328fafd9b2ad152f0afc16176cf01e1e66669
This is useful for debugging and testing.
model: Add initial SQL migration2023-12-27T11:42:25Zxengineeringme@xengineering.eu2023-12-27T11:39:10Zurn:sha1:e7bbadc8f469d5cbfbc012932c83d1d440b6e8a3
This provides the basic table structure to the database.
model: Implement database connection2023-12-27T11:27:53Zxengineeringme@xengineering.eu2023-12-27T11:14:43Zurn:sha1:c452ca52b9c18303045db838c6cead62c1454172
The already implemented storage folder should contain a sqlite database
to store most parts of the Ceres user data.