We study equivalence checking of probabilistic programs, a fundamental problem which arises in many application areas including cryptography, privacy, algorithmic fairness and machine learning. The programming language we consider manipulates polynomials over a finite field of characteristic $q$ and supports sampling and conditioning making it sufficiently expressive to encode boolean and arithmetic circuits. We consider two variants of the equivalence checking problem: the first one considers an interpretation over the finite field $Fq$, while the second one, which we call universal equivalence, verifies equivalence over all extensions $Fqk$ of $Fq$. The universal variant typically arises in provable cryptography when one wishes to prove equivalence for any length of bitstrings, i.e., elements of $F_2^k$ for any $k$. While the first problem is obviously decidable, we establish its exact complexity which lies in the counting hierarchy. To show decidability, and a doubly exponential upper bound, of the universal variant we rely on results from algorithmic number theory and the possibility to compare local zeta functions associated to given polynomials. Finally we study several variants of the equivalence problem, including a problem we call majority, motivated by differential privacy.