Skip to content
This repository has been archived by the owner on Jul 28, 2021. It is now read-only.

RoseDB - v1.2.0: Shana, The Star.

Latest
Latest
Compare
Choose a tag to compare
@ShindouMihou ShindouMihou released this 22 Jun 05:01
· 8 commits to master since this release
v1.2.0
d2d6b09
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

RoseDB: Shana, The Star.

The second major release of RoseDB after v1.1.0 which only lasted for 5 days. This version implements a lot of critical and important features for the application.

What's new?

  1. RoseDB is now fully utilizing its own query format which we call Queria, examples are found on #11 and also on a section below here.
  2. Authorization is now stronger than ever with implementation of both Layer 1 (Authorization header x Hashed Authorization) and also Layer 3 which is SSL SUPPORT
  3. Fixes issue with a condition where items are being revived after having been dropped (this is due to < v1.1.0, dropping collections/database won't actually remove anything that was on enqueued to be added).
  4. Configuration file is now beautified. #8
  5. Fixes write issues with paths (specifically making directories).
  6. Fixes issue with versioning where versions weren't being dumped properly (they were being dumped as literal folders... yes).
  7. Development snapshot versions will no longer check for updates (since the API considers dev-snapshots as old versions).
  8. Added internal locking which allows threads to communicate with one another properly over who is writing and who is reading.
  9. Moved from io.javalin to TooTallNate's websocket server which SHOCKINGLY reduced the file size up to 50%.
  10. Almost all methods on the source code are now documented, hope it will help future contributors.
  11. Major improvements on write and read methods.
  12. A major code organization was done.
  13. Fixes issue with no value Queria requests not being accepted by server.
  14. All additions to the database from now (and modifications) will be written down on a journal, and if the write to the journal fails then the transaction WILL FAILL and the client who sent the transaction will receive an error with the kode field at 0.
  15. There is now a rate limit of 3 connections per 10 seconds for each client, this helps lower CPU usage from each connection since each connection will have to hash the Authorization field of the client entering and compare it with the hash of our Authorization.
  16. The reads and writes buffer size has been reduced from 65536 to 8192, this change is still experimental and may change again in the future.
  17. Adds to the journal has a timeout of 30 seconds, if the data is still not added to the journal by then, the transaction will be considered a failure and the future will be canceled.
  18. Writes and reads have a retry limit of 500 attempts with a 50 millisecond delay between them, this retry limit involves when a Thread locks onto the file and also when a Thread writes onto a file, the timeout is at 25 seconds.
  19. The journal is read every startup to ensure that all transactions were written properly, any transactions left inside the journal will be added before being removed from the records.
  20. Writes are now synchronous for journals but for writes that are heading to the queue (and not the journal), they are done asynchronously.
  21. Fixes issue with unique field being appended to value.
  22. DELETE, DROP transactions by the time when sent to the client has already been executed (may cause performance degradation but durability > performance).
  23. Additional fixes to ListenerManager and some listeners that didn't guarantee completion of the request transaction.
  24. pom.xml will now build RoseDB.jar instead of RoseDB-v{version}-jar-with-dependencies.jar.

How to use SSL?

To use SSL, there needs to be several conditions.

  1. You must have a domain registered (for example, mihou.pw).
  2. You must have an LetsEncrypt SSL certificate.

If you have all of them, please proceed to the next step which is to create an ssl.json file which looks like this:

{
  "pathToPEM": "/path/to/pem",
  "keyPassword": "SSL KEY PASSWORD"
}

After that, restart your RoseDB server and it should attempt to use SSL for the server.

What is Internal Locking?

Internal Locking is, simply, a fancy way of saying I AM WRITING THIS FILE, NO ONE SHOULD READ THIS FILE YET! or I AM READING THIS FILE, NO ONE SHOULD WRITE YET! which allows the threads to coordinate with one another over when to write and when to read, avoiding threads where the results are broken.

This does not prevent external processes from causing mayhem though, FileLocks is needed for that but the current implementation of write and read proves to be a challenge to make FileLocks for me.

Queria

What is Queria and how does it look like?

  • Queria is the official query format of RoseDB which is extremely similar to MongoDB's query format in many ways which allows both users from RoseDB to be familiar with queries of MongoDB and also MongoDB users to be already familiar with the queries of RoseDB.

How does Queria look like?

database.collection.get(identifier, {"unique":"Unique value"})

A Queria request that does not need a unique field will look like this:

database.collection.get(identifier)

A Queria multiple field update/delete request will look like this:

database.collection.update(identifier, {"key":["key1","key2"],"value":["value1","value2"]})

A Queria add request will look like this:

database.collection.add(identifier, {"someKey":"someValue"})

In a sense, the format is database.collection.method(identifier, value), please note that value can also contain unique field which is a protected field that will be removed from the value during write to file and addition to cache. This unique field will be used for callbacks to the client.

Important notes

  1. Queria will NOT AND NEVER SUPPORT DATABASES/DIRECTORIES THAT HAS . or , or ( or ) ON THEIR NAMES.
  2. The reason for the change above is because we are using databases and collections with . as a protected RoseDB directory, for example, storing the hash token.
  3. We will be dropping support for JSON requests on v1.3.0.

Notes about Queria

Queria is still experimental and will undergo several changes along the way but the basic concept is the same, all the value will always be in JSON since RoseDB will still store data as JSON files.

How to install?

You can quickly install RoseDB by downloading the jar file below and placing it on its own folder.

Assets 3