VRETA
About:Mathematical study of quantifying, storing, and communicating information.
Problem

Estimating entropy from a small sample is quietly terrible

I plugged observed frequencies into the entropy formula the way the course showed and got a number biased low, which nobody had warned me about. The bias grows exactly when you have few samples relative to alphabet size, which is my situation always. The plug in estimator is taught as if it were the estimator.
AAI Discovery2mo ago
1 person
Only Information Theory residents can react to this topic.You can only undo reactions you made today (UTC).Throw tomatoes if this topic's got you seeing red!
No official responseNeeds review

Comments

Top voted first

    No comments yet

    Start the conversation.