ActInf GuestStream 067.1 ~ Andrés Corrada "The logic of NTQR evaluations of noisy judges"

"The logic of NTQR evaluations of noisy judges: Complete postulates and
logically consistent error correlation" Andrés Corrada dataengines.com
Whenever we evaluate N judges with T tests that have Q questions with R
responses, we can build complete sets of algebraic postulates connecting
frequencies of their aligned decisions to statistics of their correctness in
the test. This allows one to build complete theorem provers that can validate
the logical consistency (but never soundness) of any grading algorithm that
does not have access to the answer keys for the exams the judges took. Data
streaming algorithms like Good-Turing frequency smoothing are introduced to
discuss the notions of data sketches, sample statistics of the stream, and
representation free estimation of unknown or unseen stream statistics. The
case of binary classification, the (N,T=1,Q,R=2) tests, has been solved
exactly using techniques from algebraic geometry. We use the example of binary
classification to highlight how one can strip the semantics from any
evaluation in unsupervised settings and just validate the logical consistency
of the judges. A geometric viewpoint of the evaluation of R=2 tests
illustrates what can and cannot be achieved by logical consistency tests
alone. We compare some evaluations of tests on some public datasets to
illustrate how the formalism works and how easy it is to have near real time
computation with it. Active Inference Institute information: Website:
https://activeinference.org/ Twitter: / inferenceactive Discord: / discord
YouTube: / activeinference Active Inference Livestreams:
https://coda.io/@active-inference-ins...

CSID: 960023ef1269e891

Content Managed by ContentSafe.co