Gating ML Compliance

GatingML 2.0 is a flow cytometry standard that is intended to allow for a common representation of flow cytometry gates, and interoperability between different hardware and software vendors. Software that implements the standard must correctly perform several calculations, including gate membership, compensation, and transforms.
Floreada.io has near complete Gating ML 2.0 support.

There are several official compliance tests released with the GatingML standard, which allow for software vendors to test that they are performing all calculations correctly. There is also an implementation of GatingML in R, written by some authors of the standard, which is basically the de facto reference implementation of Gating ML. In the R package they provide additional compliance tests.

Unfortunately for the flow cytometry community, the stated goal of the standard - better software interoperability - has not been realized. Most software vendors, do not support the entire standard, or implement it incorrectly. Even the R implementation, written by the standard's authors does not fully implement it.

At Floreada.io, we believe we have implemented the GatingML 2.0 standard more completely then nearly any other flow cytometry analysis software. Implementing the standard was important to us for two reasons. First, we hope to set a trend that others in the flow cytometry software community follow, and create a better environment for sharing and interoperability between software. And second, by implementing the standard, it allows us to show the accuracy of Floreada.io. In order to pass the multitude of compliance test gates, all gate, compensation, transform, and ratio parameter calculations must be performed correctly and must not be off by even one event in a gate. This is an objective standard for flow cytometry analysis software accuracy which allows us match up the accuracy of Floreada.io against all other analysis software. We think the results speak for themselves.

Official Compliance Tests

There are two sets of official GatingML 2.0 compliance tests which can be obtained here.Each set of tests consists of an FCS file and a gating ML file which should be applied to it. The standard authors also provides what they believe the correct result should be.

We compare the official result for each gate with the result of the gate when applied to the FCS file using various analysis software implementations. Feel free to check the results in Floreada.io yourself. You will need to manually open the FCS file for each compliance set and apply each gate with the appropriate compensation.

Compliance Set 1

GateSpecflowCore (R)Floreada.io
Range1440440440
Rectangle1252252252
Rectangle2252252252
Polygon11582157811582
Ellipse1203203203
Ellipsoid3D419141914191
Range24710247704770
Polygon2183183183
FL2P-FL4P620620620
FL2N-FL4P238238238
FL2N-FL4N514851485148
FL2P-FL4N736173617361
Polygon3NS132513251325
RatRange1767976797679
RatRange2339833983398
RatRange1a786578657865
FSCN-SSCN3983401401
FSCD-FL1P297829782978
FSCD-SSCN-FL1N7553764764
FSCN-SSCP-FL1P595959
FSCP-SSCN-FL1N963101101
AND156145674568
AND2121212
OR1198319831983
AND3120120120
NOT1131641316413164
AND4120120120
OR2828382838283
Polygon4716716716
Rectangle3644664466446
Rectangle4127512751275
Rectangle5130313031303
ScaleRange1842584258425
ScaleRange285051278512785
ScaleRange3318131813181
ScaleRange4250925092509
ScaleRange5184018401840
ScaleRange6835183518351
ScaleRange1c691669166916
ScaleRange2c78951199211992
ScaleRange3c230923092309
ScaleRange4c187318731873
ScaleRange5c143614361436
ScaleRect1809809809
ParAnd2121212
ParAnd3120120120
ScalePar15585430430
ScaleRange6c664741134113
ScaleRange7c124785328328
ScaleRange8c626362636263
Polygon3*716716
RatRange1Bound*1066010660
RatRange1aBound*78657865
ScaleRange1Bound*85048504
ScaleRange2Bound*1336113361

1 - When you install the flowCore R package it comes with these GatingML compliance tests. However in the xml file in the Polygon1 gate definition they note that flowCore does not perform the floating point calculations correctly for this gate and so they "cheat" and change the gate definition to get flowCore to come up with the right answer. Basically flowCore messes up the calculation when events are right on the polygon gate and so they change the polygon gate vertices from [(5,5), (500, 5), (500, 500)] to [(5,5), (500, 5), (500, 500), (499.999999, 500), (5, 5.000001)] so a few events are counted correctly. If you test flowCore against the actual compliance tests in the official standard, it gets the wrong answer.

2 - The spreadsheet of results provided in the spec lists the result of this gate as 4710. However flowCore and Floreada.io both get 4770. We think the spec is incorrect. For example, for event #6691, the time value is 26.875, which obviously should be included in a range gate with bounds between 20 and 80, yet the official spec does not include this point in the gate. Given that flowCore, the de facto reference implementation, gets the same result as us, we believe 4770 is the correct answer for this test.

3 - Again quadrant gate results listed in the spreadsheet included in the spec are likely listed incorrectly, as both Floreada and flowCore get a different result.

4 - The AND1 gate test is a boolean gate which should be the result of Range2 and Polygon1 gates. The spec likely has the wrong answer for Range2 (see *2) and flowCore gets the incorrect result for Polygon1 (see *1). Therefore both get this AND gate incorrect as well. If you run the test with flowCore using their "cheat" version of Polygon1, flowCore and Floreada.io get the same result, so we are confident that we are correct.

5 - These gates all use a hyperlog transform that had a typo in the original spec. The results listed in the spec spreadsheet use a hyperlog transform with A = 0. However the actual transform definition in the test file have the Hyperlog transform defined with A = 1. Therefore the spec spreadsheet values are incorrect. The version of the compliance tests including in the flowCore library, make note of this typo in the original compliance tests.

6 - These tests are included in the flowCore compliance tests but not in the tests included in the official compliance tests.