Connecting to any data source using a custom parser

One of the DLL’s main advantages is the possibility of implementing a custom parser to visualize data from any data source.

This guide describes how to create and use a custom parser.

Note This guide assumes you have knowledge of core C# concepts. Learn about them first before continuing with this guide.

Implementing the custom parser

This tutorial is based on our implementation of the custom parser, which is a part of our sample ASP.NET Core application with Flexmonster.DataServer.Core.dll. The custom parser can be found in the CustomParser.cs file.

To create a new ASP.NET Core application with the Data Server or to embed the DLL in an existing project, refer to the guide on Flexmonster.DataServer.Core.dll.

Step 1. Initialize the custom parser

Step 1.1. Create the CustomParser class that implements the IParser interface.

Step 1.2. Specify the custom parser’s properties:

public string Type { get; private set; }

public IndexOptions IndexOptions { get; set; }

public Dictionary<int, ColumnInfo> ColumnInfoByIndex { get; set; }

To learn more about these properties, see the IParser interface section.

Step 1.3. Set the parser’s type in the constructor (in this case, the type is "custom"):

public CustomParser()
{
    Type = "custom";
}

Step 2. Specify the data

Add the field's names and types to the ColumnInfoByIndex collection (e.g., in the FillDataInfo() method):

private void FillDataInfo()
{
    ColumnInfoByIndex = new Dictionary<int, ColumnInfo>();
    ColumnInfoByIndex.Add(0, new ColumnInfo("Country", data[0, 0].GetType()));
    ColumnInfoByIndex.Add(1, new ColumnInfo("Price", data[0, 1].GetType()));
    ColumnInfoByIndex.Add(2, new ColumnInfo("Date", data[0, 2].GetType()));
}

In the example above, our parser uses built-in data defined in the data variable.

Step 3. Parse the data

The Parse() method is responsible for parsing data. In the example below, Parse() calls the FillDataInfo() method first, then processes and returns the data:

public IEnumerable<object[,]> Parse()
{
    FillDataInfo();
    for (int i = 0; i < data.GetLength(0); i++)
    {
        var partData = new object[1, 3];
        for (int j = 0; j < data.GetLength(1); j++)
        {
            partData[0, j] = data[i, j];
        }
        //return data by line
        yield return partData;
    }
}

See the IParser interface section to learn more about the Parse() method.

Step 4. Register the created parser

To be available to the DLL, the custom parser must be added to the dependency injection container in the Program.cs file:

using Flexmonster.DataServer.Core.Parsers;

var builder = WebApplication.CreateBuilder(args);
// Other configurations

builder.Services.AddTransient<IParser, CustomParser>(); 
// Other configurations

The AddTransient() method is used to provide a new parser instance each time it is requested from the service container.

Step 5. Use the parser

Now you can use the parser to create an index in the appsettings.json file. Specify the parser's type from step 1.3 in the Type property:

"DataSources":
[
  {
    "Type": "custom",
    "Indexes": {
      "custom-index": null
    }
  }
]

Notice that the "custom-index" property is set to null. This is because configurations for indexes that use a custom parser must be set in the Program.cs file.

Setting custom index options

To specify custom index options, create a class based on the IndexOptions class (e.g., CustomIndexOptions):

public class CustomIndexOptions: IndexOptions
{
  public string CustomOption { get; set; }
  // Other options

  public CustomIndexOptions(
    string customOption,
    // Other options
  ): base("custom") // The parser's type
  {
    CustomOption = customOption;
    // Other options
  }
}

You can set custom index options only in the Program.cs file:

var builder = WebApplication.CreateBuilder(args);
// Other service configurations

builder.Services.Configure<DatasourceOptions>(options =>
{
  // Other configs

  options.Indexes.Add("custom-index", new CustomIndexOptions(
    "customOption",
    // Other options
  ));
  // Other configs
});
// Other service and app configurations

The created custom options should be handled in the custom parser. For example, in the Parse() method:

public IEnumerable<object[,]> Parse()
{
  var customIndexOptions = (CustomIndexOptions)IndexOptions;
  if (customIndexOptions.CustomOption == "customOption")
  {
    // Code to run if the custom option is set
  }
  // Other code
}

The IParser interface

All the parsers should implement the IParser interface. IParser has the following definition:

public interface IParser
{
    string Type { get; }
    IndexOptions IndexOptions { get; set; }
    IEnumerable<object[,]> Parse();
    Dictionary<int, ColumnInfo> ColumnInfoByIndex { get; }
}

Below is a detailed description of the properties:

Property/Type	Description
Type String	The parser's name. It will be used in the `appsettings.json` file when creating an index.
Parse Function	Responsible for parsing the data. The purpose of the function is partial data loading, which allows working with large datasets. The `Parse()` method has the following signature: `IEnumerable<object[,]> Parse();` `object` can have either any primitive type or the DateTime type, so all dates in the dataset to parse should have the DateTime type.
IndexOptions IndexOptions	Stores the options defined when creating an index. Index options for a custom parser are set inside the dependency injection container in the `Program.cs` file. The `IndexOptions` is a class from the DLL that corresponds to the IndexObject configuration for the Data Server DLL. Can be used to create custom index options.
ColumnInfoByIndex Dictionary<Number, ColumnInfo>	Allows setting the name and type for columns. `ColumnInfoByIndex` is a Dictionary, where key is the column’s index (starting from `0`) and value is an instance of the ColumnInfo class. Note that the `ColumnInfoByIndex` collection should contain information about columns before the `Parse()` method returns the data.

ColumnInfo

Property/Type	Description
Name String	A column's name.
Type Type	Type of data stored in a column.

What's next?

You may be interested in the following articles: