Conversation
Using drizzle
* add components table * improve typing for json columns * fix non-nullable columns * implement basic entity generation functions
* Implement repository method * Test that deletes cascade to relations
|
RE pglite (#593): the db is excellent, would provide ample headroom for scaling MS in the future, and has similar performance where it counts....but it turns out this all comes with the trade-off of a massive increase in memory usage. Like, 3-4x more memory than 0.13.3 and the sqlite implementation. I guess this should have been expected since it's basically an in-memory instance of postgres but I was still hoping for a smaller footprint. As detailed here in even the best case scenario with their bleeding edge stuff is still 128MB increase. I was seeing ~400MB after MS settled into normal operations after startup. And ~700MB at peak on startup. Compared to ~300 startup -> ~150 with sqlite. Maybe this can be reduced in the future. And pglite is an awesome technology that I hope to adopt for other projects. But too many users expect a low memory footprint from MS -- I think many people run it on low-power devices like raspis. So back to |
Lots of cleaned up methods and better framing if db backend ever becomes async again
|
Just went ahead and updated my instance. No problems so far, but it looks like this branch has the artist refactor (1b53d22). Should artist MBIDs be written to sources on this branch? I'm not seeing them for teal.fm. From what I can see, the teal.fm play record should support both. Logsteal.fm record |
|
@owendaprile should be implemented now. The initial commit was just getting MS working with the new structure, I never went back to actually leverage mbids where they were provided. |
|
@4rft5 have you tried updating recently? I cannot reproduce the scrobbles page being stale. |
I've been following along and updating as new releases are made for the PR but have the same result, all clients It might've just been a fluke with my database. I've removed it and restarted the container and it looks like it's made a new one and is counting them again. 
|
|
The latest builds (c86819b and later) contain a db migration and re-run old cached scrobble conversion. Your old, cached scrobbles should already have been converted when you started using this pr, so you should see no functional change. But you should see this happen, regardless:
If you don't see this happen or have any issues with using the newest image, please report it. |
…pass ignored response Fixes #596
|
I just updated and tested that LFM 30s fix and it's working. Still no more issues with the stale database. I also went back and checked and it did run the migration and make the backup. All looks good! |
|
Out of curiosity when the PR finally gets merged will it include the UI overhaul or has that been pushed back? Had a look for the PR regarding the UI overhaul and couldn't find it anymore but I might be blind o7. Regarding the current DB changes though I haven't noticed any issues since updating to it. Been working great, my discord RPC has been a bit spotty when listening to repeating songs but I'm fairly sure that's unrelated. Big fan of the DB change personally!! |
|
The UI is being tracked in issue #500 , no pr yet. There is a storybook folder that has all the current prototype work in it, in master/this pr. I got a bit into it and then realized I really needed the DB to make it fully useful. Without it there would just be too much kept in memory. So after db gets merged ill be able to resume work on the UI. Re: repeats, I think someone else has an issue open for repeat issues as well. I'm going to postpone looking into that until after db is released as this change is big enough as it is. Thanks for the feedback! I've been running it in prod for awhile now too. It's almost ready to rebase/squash and merge. |
Right that explains why I couldn't find it hehe. That is a fair reason to put it on hold. Will be exciting when it comes out I shall say, the photos of it I've seen so far look really cool.
Yeah it's not a huge deal to me personally, my songs scrobble to my Koito server fine I just noticed my Discord RPC would just not display randomly. Which personally isn't a big deal I just have it because it's fun to make things fancy. |
* Granular persistence of successfully migrated scrobbles * Don't delete cache after processing * reduce logging noise and improve wording * update upgrade wording wrt cache migration
* Use ErrorLike or ErrorIsh for error utilities, rather than requiring object be an Error instance * Render error summary on dead scrobble page and have copy action include all error data * serialize errors when sending through api * fix error assignment for new dead scrobbles
📦 A new release has been made for this pull request.To play around with this PR, pull an image:
Images are available for x86_64 and ARM64.
|
The new ui is using Chakra UI for the framework and it has pretty granular theming functionality. I haven't looked into it much but it looks like its pretty easy to create custom color palettes. The only drawback is that the frontend is bundled when the docker image is built so any themes would need to be built into the source rather than being user-supplied at container runtime. It looks like chakra can work with multiple preset themes but it will require some work since it's not an out-of-the-box feature Here's a more involved implementation, for my future reference or anyone else who wants to implement this and submit a pr: https://www.crema.us/blog/make-a-custom-color-theme-switcher-with-chakra-ui-and-semantic-tokens |
|
Progress so far has been released as 0.14.0-beta.1 and a new issue has been opened to track any problems. Please move further discussion to the new issue, thanks! |
8 participants
Checklist before requesting a review
Type of change
Please delete options that are not relevant.
Describe your changes
This PR represents the preliminary changes for multi-scrobbler
0.14.0. It is a major update that touches almost every aspect of MS in some way. The main feature is a database implementation to persist data, however since it will require breaking changes I've folded in other major/breaking changes so that the bandage can be ripped off all at once with this update. Change highlights include:*_ID=and*_NAME=*_ENABLE=for all clients/sourcesThe DB implementation with queues has additional benefits:
From a storage perspective it's also pretty compact:
60kb160kb1mb356kb3.1mb680kb6.4mbThis PR is already in a good state for daily usage. I am running it on my own server in "production". There are likely still bugs and config kinks to work out. I need the help of MS users to test out the built PR docker image and report how well it works, or what issues, you run into.
Usage
foxxmd/multi-scrobbler:pr-588and redeploy