CompressionStyle.RecordLevel appears to be unimplemented for PhysicalParser?

[SRM256]

I have some PQD files that I want to use your library to read. I am getting ErrorElements generated when I read it. I compared the behavior of your code to a C library for reading PQD files. The difference appears to be that your code doesn't implement record-level decompression. The only use of CompressionStyle.RecordLevel that I can find is in the PQD writer code, not in the reader code. Is my analysis correct?

I tried running your PQDIFDump.exe and got the same errors, so I'm pretty sure it isn't something I'm doing wrong.

A file that shows the problem is the first example file on the IEEE standards page:
https://grouper.ieee.org/groups/1159/3/docs/20020331-092021.pqd

Please advise me on how to handle these files.

[StephenCWills]

The LogicalParser sets the compression settings on the PhysicalParser based on what it reads from the ContainerRecord in the PQDIF file.

gsf/Source/Libraries/GSF.PQDIF/Logical/LogicalParser.cs

Lines 122 to 133 in 4d453a3

	/// <summary>
	/// Opens the parser and parses the <see cref="ContainerRecord"/>.
	/// </summary>
	/// <exception cref="InvalidOperationException"><see cref="FileName"/> has not been defined.</exception>
	/// <exception cref="NotSupportedException">An unsupported compression mode was defined in the PQDIF file.</exception>
	public void Open()
	{
	m_physicalParser.Open();
	m_containerRecord = ContainerRecord.CreateContainerRecord(m_physicalParser.NextRecord());
	m_physicalParser.CompressionAlgorithm = m_containerRecord.CompressionAlgorithm;
	m_physicalParser.CompressionStyle = m_containerRecord.CompressionStyle;
	}

The PhysicalParser doesn't do this itself because it's designed to be unaware of the logical structure of the PQDIF file. Therefore, if you are using the PhysicalParser directly, you'll need to set these properties manually (after reading first record which should always be uncompressed).

FYI, it looks like the PQDIFDump tool does the bare minimum level of effort to read an uncompressed PQDIF file.

---

Finally, as for the question of whether RecordLevel compression is supported, it should be. It's actually TotalFile compression that is not supported.

gsf/Source/Libraries/GSF.PQDIF/Physical/PhysicalParser.cs

Lines 180 to 197 in 4d453a3

	/// <summary>
	/// Gets or sets the compression style used by the PQDIF file.
	/// </summary>
	/// <exception cref="NotSupportedException"><see cref="CompressionStyle.TotalFile"/> compression has been deprecated by the standard and is not supported by this parser.</exception>
	public CompressionStyle CompressionStyle
	{
	get
	{
	return m_compressionStyle;
	}
	set
	{
	if (value == CompressionStyle.TotalFile)
	throw new NotSupportedException("Total file compression has been deprecated and is not supported");
	m_compressionStyle = value;
	}
	}

[SRM256]

Thanks.

"Therefore, if you are using the PhysicalParser directly"

This hints to me that I should instead use the LogicalParser? (I had not found that class before.)

Do you have a recommended example program that I can look at for the use of this library to read a whole PQDIF file?

[LATER EDIT] I did change over to LogicalParser, and that only exposes the Observation records, not all the records. But at least now I have all the observations. It seems strange that there does not appear to be any way to iterate all the records.
[EVEN LATER] Ah ha. I found that there is a separate list for the DataSourceRecords. a) Why are the data source records exposed as a list property but the observation records exposed with an iterator method style? b) Why is there no exposed property for all the records together?
I'm slowly figuring out how this API gets the information out of the file. :-)

[SRM256]

I have some documentation changes to recommend that I think would help future explorers like myself gain an entry point to this code base.

For PhysicalParser:

    /// <summary>
    /// Represents a parser which parses the physical structure of a PQDIF file.
    /// </summary>
    /// <remarks>
    /// If your goal is to read a PQD file into an application, you do not want to 
    /// instantiate <see cref="PhysicalParser"/> directly. Instantiate
    /// <see cref="GSF.PQDIF.Logical.LogicalParser"/> instead.
    /// </remarks>

For LogicalParser

    /// <summary>
    /// Represents a parser which parses the logical structure of a PQDIF file.
    /// </summary>
    /// <remarks>
    /// This class makes the complete contents of the PQD files available to 
    /// callers. Use the iteration of <see cref="ObservationRecord"/> to examine
    /// each of the the measurements recorded in the file. The <see cref="DataSourceRecord"/>
    /// and <see cref="MonitorSettingsRecord"/> for each observation is exposed as
    /// a property on each observation record. The same data source and monitor settings
    /// records may be referenced by multiple observation records.
    /// </remarks>

[StephenCWills]

You're moving a bit too quickly for me to answer everything, but that's good! I'm glad you're making progress.

You may not need this anymore, but here is an example of usage for the LogicalParser.

https://github.com/GridProtectionAlliance/openXDA/blob/d1b99170e3932c8a45b62de1b92aeade25129309/Source/Libraries/FaultData/DataReaders/PQDIFReader.cs#L122-L138

---

Why are the data source records exposed as a list property but the observation records exposed with an iterator method style?

The idea behind the logical parser was to apply the rules governing the logical structure of the file. According to the standard, observation records should be associated with the nearest preceding data source record in the linked list. The ObservationRecord class has a DataSource property that provides the appropriate DataSourceRecord according to the logical structure, thus allowing the parser to resolve the relationship between these records without any additional hassle for the user.

The list of data source records is built while parsing the observation records and can be accessed after you've finished reading through the file in order to detect potentially malformed sequences. Specifically, if a data source record is immediately followed by another data source record, the first record would otherwise be undetectable when using the logical parser. Note, this can also be the case for monitor settings records, but I didn't have a use-case for detecting those so I don't give them the same treatment.

---

Why is there no exposed property for all the records together?

To get the full sequence of records as they are defined according to the physical structure, you should use the PhysicalParser. The linked list is mostly irrelevant to the LogicalParser since it's more concerned with appropriately associating the data source records and monitor settings records with each observation record.

---

I have some documentation changes to recommend that I think would help future explorers like myself gain an entry point to this code base.

Good idea! These classes certainly could use some remarks to clarify the differences between them. I may elaborate a bit beyond your suggestion, but I will work on incorporating these into the codebase. Thanks very much for the feedback.

[StephenCWills]

FYI, d7abb17