Recommended use in ASP.NET Core applications

[harrison314]

Thank you for interesing project! I am thinking of creating a (experimental) timeseries database on top of ZoneTree.
I'm in the design phase and have a few questions.

What is the recommended use in multi-thread/async/multi-user applications (eg. ASP.NET Core)? Create a new instance of the ZoneTree class per request or have it as a singleton?

If ZoneTree is to be a singleton, should maintenance (IZoneTreeMaintenance.CompleteRunningTasks()) be started at regular time intervals or should it be done after a certain number of entries?

I think it would be good to mention the answers to these questions in the documentation.

[koculu]

Thank you for your interest in the project! I'm glad to hear you're considering using ZoneTree for your timeseries database. Here are the answers to your questions:

First Question:

ZoneTree can be used in various patterns depending on your application's requirements. It is thread-safe, meaning you can have a single ZoneTree instance accessible from all threads without needing additional synchronization.

Second Question:

The ZoneTreeMaintainer class is responsible for maintaining a ZoneTree instance. Here are the key events and tasks it handles:

• OnMutableSegmentMovedForward Event:

  • Initiates merging if ReadOnlySegmentsCount and ReadOnlySegmentsRecordCount meet the desired criteria.

• OnDiskSegmentCreated Event:

  • Initializes a sparse array on a disk segment to reduce disk I/O.

• OnMergeOperationEnded Event:

  • Manages the result of a StartMerge request.

• OnZoneTreeIsDisposing Event:

  • Completes running merge operations before disposing of the ZoneTree.

• Periodic Task:

  • Periodically cleans up the disk segment read buffers (default: once every 5 seconds). You can disable the periodic task to keep data in memory forever to speed up reads. Trade-off between memory usage and read speed.

You can integrate this maintainer in an ASP.NET Core application using the IHostedService interface. Below is a sample implementation:

public sealed class ZoneTreeMaintainerHostedService<TKey, TValue> : IHostedService, IDisposable
{
    private readonly ZoneTreeMaintainer<TKey, TValue> _zoneTreeMaintainer;

    public ZoneTreeMaintainerHostedService(IZoneTree<TKey, TValue> zoneTree)
    {
        _zoneTreeMaintainer = new ZoneTreeMaintainer<TKey, TValue>(zoneTree, false);
    }

    public Task StartAsync(CancellationToken cancellationToken)
    {
        // Note:
        // Periodic Timer periodically cleans up the disk segment read buffers (default: once every 5 seconds).
        // You can omit the periodic timer to keep data in memory forever to speed up reads. 
        // The trade-off between memory usage and read speed.
        _zoneTreeMaintainer.EnablePeriodicTimer = true;
        return Task.CompletedTask;
    }

    public Task StopAsync(CancellationToken cancellationToken)
    {
        _zoneTreeMaintainer.EnablePeriodicTimer = false;
        _zoneTreeMaintainer.CompleteRunningTasks();
        return Task.CompletedTask;
    }

    public void Dispose() => _zoneTreeMaintainer?.Dispose();
}

builder.Services.AddSingleton((serviceProvider) =>
{
    return new ZoneTreeFactory<string, string>().OpenOrCreate();
});

builder.Services.AddHostedService<ZoneTreeMaintainerHostedService<string, string>>()

[harrison314]

ZoneTree can be used in various patterns depending on your application's requirements. It is thread-safe, meaning you can have a single ZoneTree instance accessible from all threads without needing additional synchronization.

When instantiating ZoneTree per request, will ZoneTree prove to access files from multiple instances fully in read-write mode, or are there any limitations?

You can integrate this maintainer in an ASP.NET Core application using the IHostedService interface.

Thanks for the code.

[koculu]

I don't fully understand your question. Could you clarify why you are instantiating a ZoneTree per request? If you have one ZoneTree per request, why are multiple instances a concern?

Note:

ZoneTree is an LSM Tree (Log-Structured Merge Tree). This means that writing is append-only and files are never directly edited, which helps avoid common concurrency issues.

As far as I know, there are no specific limitations regarding multiple instances.

For more information, you can read about LSM Trees here.

[harrison314]

I was thinking of having a separate ZoneTree database for each bucket. But according to your answers, it makes no sense to think about it and it is more efficient to use ZoneTree as a singlton.