Where it looks like the output is possible written underneath each State? I am not sure if I should just apply the number written under the State (e.g A/0) to each of the edges pointing away from the state to make it look like the first diagram or if the number means something different.
There are two types of state machine, Mealy and Moore. The difference between the two is that while outputs in Moore machines depend only on the current state, Mealy machine outputs also depend on the current inputs as well.
Your first example is how we describe a Mealy machine. The bubbles represent the current state, each of which has a name (lets call it \$S\$). The arrows represent the transitions. The labels on the arrows are in the form \$(I/O)\$ where \$I\$ is the input value, and \$O\$ is the new output value that would result from the corresponding transition.
The second example is a description of a Moore machine. Again the bubbles represent the current state, and arrows the transitions. However the labelling is different. The labels for the transitions are now just \$(I)\$ - i.e. the input value. The label for the state name is now \$(S/O)\$ - the state name, followed by the output in that state.
Because the Moore machine output depends only on the state, the output values are labelled in the state bubble. Whereas because the outputs in the Mealy machine are dependent on the input and react immediately, they will change on the transitions, and as such are labelled on the transition arrow.