Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get size of acid-state? #124

Open
CSchank opened this issue Apr 16, 2019 · 3 comments
Open

Get size of acid-state? #124

CSchank opened this issue Apr 16, 2019 · 3 comments

Comments

@CSchank
Copy link

CSchank commented Apr 16, 2019

I am wondering if there is a good / reliable way to find out how much space a given acid-state instance is taking up? Would finding the size of the latest checkpoint be a good measure of the memory taken by that instance? If there isn't a good way to do this currently, perhaps it could be added to the library? We'd like to be able to easily monitor how big our databases are getting.

Thanks for your help.

Best regards,
Chris

@lemmih
Copy link
Member

lemmih commented Apr 16, 2019

You could use Data.Compact: http://hackage.haskell.org/package/compact-0.1.0.1/docs/Data-Compact.html

compactSize <=< compactWithSharing :: a -> IO Word

@adamgundry
Copy link
Contributor

That's a nice trick, albeit potentially expensive.

The size of the latest checkpoint on disk is probably going to be roughly proportional to the heap size in most cases, although this will depend a bit on the efficiency of your in-memory vs binary representations and on how much sharing you have.

Another option in the wild is to manually define an operation to calculate the heap size of your data (e.g. hackage-server does this via the MemSize typeclass). This has the downsides that it is a bit of manual work in which you have to get the calculations right, of course, and it doesn't take account of sharing.

@CSchank
Copy link
Author

CSchank commented Apr 17, 2019

Thanks for the suggestions. The first one doesn't work with our data. We could do the second thing but the effort might not be worth it. We also tried ghc-datasize (http://hackage.haskell.org/package/ghc-datasize) but that doesn't seem to have been updated to work with newer ghc versions.

Thanks anyway for the suggestions.

Any other suggestions people have would be welcomed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants