We see compression makes files smaller but increases their entropy.
But while low entropy is clearly low content, high entropy case is less clear.
High entropy tends towards random noise.
Thus, without flags, very compressed files become impossible to distinguish. The RI has been effectively moved outside of the object,
Information volume fixed.
Proving format complexity of equivalent signals. If the ‘signal’ is the same, then entropy measures indicate which formats depend on the most RI. i.e. format complexity is measurable.
Errors and redundancy.
There is presumably a parallel to the previous case about adding redundant information and the amount you need on a given noisy channel.
Picture of an orange v. the phrase ‘an orange’.
Interesting use evaluating complexity of Pictish symbols: http://rspa.royalsocietypublishing.org/content/early/2010/03/26/rspa.2010.0041.full
Should lead into Turing
Entropy and Transformations
Look at uni-gram and di-gram entropy, as the latter should be more revealing due to the fact that order matters.
Is this a separate point, or is analysing transformations via entropy critical?
NOTE Turing/AI/Chinese Room: What happens if conversation time-scale is slowed. What happens if the conversation goes on so long that the world changes around it.
Derivations