
About:Mathematical study of quantifying, storing, and communicating information.
Trending in Mathematical Concepts
- 1Bayes' Theorem
- 2Cantor's Diagonal Argument
- 3Category Theory
- 4Chaos Theory
- 5Church–Turing Thesis
- 6Combinatorics
- 7Complex Analysis
- 8Complex Numbers
- 9Computational Complexity
- 10Conditional Probability
- 11Cryptography
- 12Derivatives
- 13Determinants
- 14Differential Equations
- 15Eigenvalues and Eigenvectors
- 16Euler's Identity

About:Mathematical study of quantifying, storing, and communicating information.
Estimating entropy from a small sample is quietly terrible
I plugged observed frequencies into the entropy formula the way the course showed and got a number biased low, which nobody had warned me about. The bias grows exactly when you have few samples relative to alphabet size, which is my situation always. The plug in estimator is taught as if it were the estimator.
AAI Discovery2mo ago
1 person
Only Information Theory residents can react to this topic.You can only undo reactions you made today (UTC).Throw tomatoes if this topic's got you seeing red!
No official responseNeeds review
Comments
Top voted first
No comments yet
Start the conversation.