Persist current revision to a SQL-lite db and only update the stack if the revision and stack content changed #110

clangenb · 2025-02-28T14:47:18Z

So our goal is to prevent unnecessary restarts as our restarts incur minimal offline time, which we absolutely want to prevent. Hence, this PR introduces the following optimizations:

Only redeploy the stack iff the git revision and the particular stack's docker compose file have changed.
Additionally, we persist both into an SQL light DB such that a restart of swarm cd won't cause a restart of all stacks.

sanzoghenzo

This is great!

I've added my personal opinions on this: functionally it is OK (though I still have to test it), but I would reword some things.

swarmcd/sql_db.go

swarmcd/stack.go

clangenb · 2025-03-04T08:49:45Z

Cool, thanks a lot for taking a look at it! I have added your requested changes. 👍

sanzoghenzo

Thank you for the edits!
these are my last ramblings, I'll let you decide if they make sense 😉

README.md

clangenb · 2025-03-06T14:17:18Z

Thanks again for the detailed review. I have added the changes according to your feedback. 🚀

sanzoghenzo

sorry for being so pedantic, but I ask you to do one last change to the docs to be as explicit as possible.

One thing I'm wondering just now, though: is it important to persist the database to disk?
Couldn't we just use an in-memory instance of the sqlite db?
I mean, the added complexity of the storage on disk will only save you a few re-deployment when you restart swarm-cd (which should be a zero operation since nothing changed), while the big benefit from this (as I see it) is to avoid the re-deployment every two minutes.

README.md

clangenb · 2025-03-07T08:07:02Z

sorry for being so pedantic, but I ask you to do one last change to the docs to be as explicit as possible.

One thing I'm wondering just now, though: is it important to persist the database to disk? Couldn't we just use an in-memory instance of the sqlite db? I mean, the added complexity of the storage on disk will only save you a few re-deployment when you restart swarm-cd (which should be a zero operation since nothing changed), while the big benefit from this (as I see it) is to avoid the re-deployment every two minutes.

No worries, being pedantic creates better code. In our case, we explicitly want to avoid redeployments due to restarts of swarm, as it will cause an unnecessary restart of the deployed service, which does incur minimal downtime - sometimes at unpredictable times.

However, maybe we could maybe disable that feature by default, and setting the env var will enable it. What do you think?

sanzoghenzo · 2025-03-07T21:22:20Z

as it will cause an unnecessary restart of the deployed service

Oh, now I'm the non-pro in this domain 😅 I thoight that re-issuing a docker stack deploy with the same exact file wouldn't result in a redeploy...

I see that there's the --resolve-image flag that defaults to always, and can be set to changed to only recreate the container if the image hash changes (be it a new tag or an update of the same tag)... maybe it can be enough to avoid or at least reduce the restarts?

This is not to turn down your contribution, it is just to analyze the situation and make it as efficient as possible (event if this is a small, simple addition, no code is the best code 🤣 )

clangenb · 2025-03-08T06:54:49Z

hahaha, no worries. I am equally interested in finding the best aka most slim solution here. Let's have @mosonyi chime in here. He is the pro, and if he agrees that your suggestion might fix our issue, then I happy.

Co-authored-by: Andrea Ghensi <[email protected]>

clangenb · 2025-03-22T11:29:34Z

Thanks to your pedantic feedback the code has been improved much more. :) It should be ready for another round of review.

swarmcd/swarmcd.go

sanzoghenzo

This looks great!

Only one note on the updateStack function: the logger is initialized to show the stack name as extra parameter, so there's no need to add it to the message... but it doesn't hurt, either 😉

clangenb · 2025-03-22T17:41:58Z

This looks great!

Only one note on the updateStack function: the logger is initialized to show the stack name as extra parameter, so there's no need to add it to the message... but it doesn't hurt, either 😉

Thanks a lot for the continuously reat feedback! Yeah, I noticed that, but I find it hard to see to which stack the logs belong to, as the logger appends the stack name instead of prepending. This is why I did it like that.

If you have a better idea, I am happy to listen. 😄

sanzoghenzo · 2025-03-22T19:04:18Z

I find it hard to see to which stack the logs belong to, as the logger appends the stack name instead of prepending. This is why I did it like that.

If you have a better idea, I am happy to listen. 😄

You're right, it is more readable, and I don't know go that well to propose better ways.

Now that all is good, I'll try to give it a spin before merging

sanzoghenzo · 2025-05-19T17:26:01Z

I'm sorry I didn't have the time to test this, and I don't believe I can spend time on this project for a while...
@m-adawi should I go ahead and merge it anyway, or would you like to give it a spin first?

sanzoghenzo

Hi there, sorry for the radio silence!

I finally got some time to test it, but I realized there still are some things I would do differently, mainly to concentrate the configuration in a single place and to avoid high coupling in the code.

sanzoghenzo · 2025-10-20T14:21:46Z

swarmcd/database.go

+func getDBFilePath() string {
+	if path := os.Getenv("SWARMCD_DB"); path != "" {
+		return path
+	}
+	return "/data/revisions.db" // Default path
+}


Should we move this to the config module?

viper could also handle environment variables, but it is not enabled yet.
It should be a matter of adding the db field to the Config struct and the following lines to LoadConfigs

configViper.SetEnvPrefix("SWARMCD") configViper.AutomaticEnv() configViper.SetDefault("db", "/data/revisions.db")

sanzoghenzo · 2025-10-20T14:29:00Z

swarmcd/stack.go

+	stackDb, err := initStackDB(getDBFilePath(), swarmStack.name)
+	if err != nil {
+		return nil, fmt.Errorf("failed to open database: %s", err)
+	}
+	defer stackDb.close()


Maybe I'm missing something here, but isn't it better to initialize the database when swarmcd starts and just pass the db to updateStack? This will be a first step towards dependency injection to perform test with a fake database, or simplify the swap to another DB if needed.

sanzoghenzo · 2025-10-20T14:31:01Z

swarmcd/database.go

+type stackDB struct {
+	db        *sql.DB
+	stackName string
+}
+
+// Ensure database and table exist
+func initStackDB(dbFile string, stackName string) (*stackDB, error) {
+	db, err := initSqlDB(dbFile)
+	if err != nil {
+		return nil, err
+	}
+
+	return &stackDB{db: db, stackName: stackName}, nil
+}
+
+func (stackDb *stackDB) saveLastDeployedMetadata(stackMetadata *stackMetadata) error {
+	return saveLastDeployedMetadata(stackDb.db, stackDb.stackName, stackMetadata)
+}
+
+func (stackDb *stackDB) loadLastDeployedMetadata() (*stackMetadata, error) {
+	return loadLastDeployedMetadata(stackDb.db, stackDb.stackName)
+}
+
+func (stackDb *stackDB) close() error {
+	return stackDb.db.Close()
+}


Not sure if this is really needed, it seems to me that it adds complexity just to avoid repeating the name of the stack on 3 LOC

clangenb · 2025-10-21T04:59:17Z

Hey there, thanks for looking into this. I have put in some considerable effort into this, which goes beyond this PR.

I guess, it is better to extract some new PRs based on my current state if there is still interest in this. :)

sanzoghenzo · 2025-10-21T05:46:07Z

Hey there, thanks for looking into this. I have put in some considerable effort into this, which goes beyond this PR.

Oh wow, now I see the other branches on your fork.. you've done a tremendous job!
Again, sorry for ignoring this 😅

I guess, it is better to extract some new PRs based on my current state if there is still interest in this. :)

Of course! Should we close this and start with a new PR? Or could you push your latest edits on this branch? (We will sqash everything in a single feature commit, so the lengthy commit history is not a problem!)

vhaudiquet · 2025-10-29T16:51:13Z

This should fix #126, and I'm really grateful for your work ! :)

clangenb · 2025-10-30T00:40:42Z

Sorry I was really busy the last two weeks. I will try to follow up soon. I guess, I will do separate PR, but keep this open for visibility until the new one is in place.

vhaudiquet · 2025-11-18T22:36:25Z

@clangenb Any updates on this? I would really love to try that! Is the version on your branch ready or does it need more work? I could build it myself in the mean time

clangenb · 2025-11-18T23:56:31Z

Thanks for remindinding me. I should have time this today. 😃

clangenb · 2025-11-20T07:04:44Z

Sorry, I could not use the time window I thought that I have yesterday. But I invite you to check downstream's develop branch: https://github.com/clangenb/swarm-cd/tree/develop

I have been using it in production since a year now without any issues. It contains this fix, and also quite some other improvements.

vhaudiquet · 2025-11-28T00:45:17Z

I have to say, thank you! I've been running your branch for a week now and it has been wonderful.
The only thing I miss is that swarm-cd used to automatically update images that I tagged :latest, which is not the case anymore. I've heard that this wasn't really a feature, and wasn't really reliable anyway, so I won't mind losing that for now. I guess that could be fully implemented as a feature later on, if needed.

clangenb · 2025-11-28T03:34:53Z

Cool, glad to hear that! I will still try to upstream this eventually, but it could be that it needs to wait until the Christmastime.

Yeah, the latest has always been kind of tricky. It should be possible to implement though, although I would still prefer to have an explicit re-pull of the image in my case. :)

clangenb changed the title ~~Persist current revision to a litesql db and only update the stack if the revision changed~~ Persist current revision to a SQL-lite db and only update the stack if the revision changed Feb 28, 2025

sanzoghenzo reviewed Mar 2, 2025

View reviewed changes

sanzoghenzo reviewed Mar 4, 2025

View reviewed changes

README.md Outdated Show resolved Hide resolved

sanzoghenzo reviewed Mar 7, 2025

View reviewed changes

README.md Outdated Show resolved Hide resolved

README.md Show resolved Hide resolved

README.md Outdated Show resolved Hide resolved

README.md Outdated Show resolved Hide resolved

clangenb and others added 21 commits March 19, 2025 14:23

[swarmcd] add stack_test.go and first unit test

342b2b8

Check git revision while updating the stack.

c10f913

add sql_db for persisting git revisions

c6f4a98

integrate persisting revision to db

6c00e94

add error handling in sql_db code

e734169

minor fixes

e02bed5

ui fixes

cfdbc31

log the current revision in stack.go

f6e033a

add repos.yaml/stack.yaml to gitignore

11562d4

fix readme regarding volume

3b40eb6

delete docker compose

03afa1f

ignore docker compose

3d84e0d

Rename revision loading/saving

0a89d74

Co-authored-by: Andrea Ghensi <[email protected]>

Rename revision loading/saving

0e0bd65

Co-authored-by: Andrea Ghensi <[email protected]>

Rename revision loading/saving

cf96bae

Co-authored-by: Andrea Ghensi <[email protected]>

Rename revision loading/saving

d1c4a4d

Co-authored-by: Andrea Ghensi <[email protected]>

rename sql_db to db

a2b7343

read db path from env var

7e7b919

add env var config to Readme

a91cadf

Fix Typo

d014319

Co-authored-by: Andrea Ghensi <[email protected]>

[Readme] remove unnecessary comments

ff61809

clangenb and others added 5 commits March 22, 2025 10:20

Update swarmcd/stack.go

a5d8999

Co-authored-by: Andrea Ghensi <[email protected]>

introduce a struct for nicer interface

98b1b06

add fmtHash method to struct

965961d

fix sprintf statement

4783741

fix null-pointer upon first startup

c40c9e5

clangenb mentioned this pull request Mar 22, 2025

Detect new stacks on the fly #115

Draft

clangenb added 7 commits March 22, 2025 11:17

persist current revision to DB even if we don't deploy

793f0c4

save deployment date

85d20e7

fix null pointer error

23b3dff

some renaming

ea13197

more renaming

7c77900

introduce stackDb abstraction

c5e9f1c

some renaming

4b800f9

clangenb commented Mar 22, 2025

View reviewed changes

swarmcd/swarmcd.go Show resolved Hide resolved

sanzoghenzo approved these changes Mar 22, 2025

View reviewed changes

sanzoghenzo requested changes Oct 20, 2025

View reviewed changes

sanzoghenzo mentioned this pull request Oct 20, 2025

429 Too Many Requests permanently with SwarmCD #126

Open

Persist current revision to a SQL-lite db and only update the stack if the revision and stack content changed #110

Are you sure you want to change the base?

Persist current revision to a SQL-lite db and only update the stack if the revision and stack content changed #110

Uh oh!

Conversation

clangenb commented Feb 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sanzoghenzo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

clangenb commented Mar 4, 2025

Uh oh!

sanzoghenzo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

clangenb commented Mar 6, 2025

Uh oh!

sanzoghenzo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

clangenb commented Mar 7, 2025

Uh oh!

sanzoghenzo commented Mar 7, 2025

Uh oh!

clangenb commented Mar 8, 2025

Uh oh!

clangenb commented Mar 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

sanzoghenzo left a comment

Choose a reason for hiding this comment

Uh oh!

clangenb commented Mar 22, 2025

Uh oh!

sanzoghenzo commented Mar 22, 2025

Uh oh!

sanzoghenzo commented May 19, 2025

Uh oh!

sanzoghenzo left a comment

Choose a reason for hiding this comment

Uh oh!

sanzoghenzo Oct 20, 2025

Choose a reason for hiding this comment

Uh oh!

sanzoghenzo Oct 20, 2025

Choose a reason for hiding this comment

Uh oh!

sanzoghenzo Oct 20, 2025

Choose a reason for hiding this comment

Uh oh!

clangenb commented Oct 21, 2025

Uh oh!

sanzoghenzo commented Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vhaudiquet commented Oct 29, 2025

Uh oh!

clangenb commented Oct 30, 2025

Uh oh!

vhaudiquet commented Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

clangenb commented Nov 18, 2025

Uh oh!

clangenb commented Nov 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

clangenb commented Feb 28, 2025 •

edited

Loading

clangenb commented Mar 22, 2025 •

edited

Loading

sanzoghenzo commented Oct 21, 2025 •

edited

Loading

vhaudiquet commented Nov 18, 2025 •

edited

Loading

clangenb commented Nov 20, 2025 •

edited

Loading