One thing I think I am seeing is that all of your chips look like they have the same footprint which is making me think they are all the same chip. If they are the same chip then all of them need to have a unique address, otherwise all of them will try to respond to the same commands given. This is somewhat fine if they all need to behave at the same time, not so great if they need to act independently. If they have the same address though, if one of them breaks, is missing, didn't get a command, etc, then your code will be unaware of this.
As for pull-ups, pipe is absolutely correct, you only want one pair of pull-ups used per bus. The picture you posted shows four completely separate busses (micro to multiplexer, multiplexer to IO expanders and LED blinkers, Hub repeater to more IO expanders, and multiplexer to data converter, eeprom, etc).
Section 7.1 of the I2C standard (www.nxp.com/documents/user_manual/UM10204.pdf) shows how to size the pull-ups. Table 10 in section 6.1 specifies the rise times for the different speeds.
One trick I have learned for correctly sizing the pull-ups is specify a mid-range resistor (~5k) as the pull-up, then scope the waveforms on the prototype unit. The actual resistor required will be equal to CurrentResistor * (TargetRiseTime / MeasuredRiseTime). A resistor smaller than the number specified will give you the rise time to meet the specifications. If the resistor is smaller than the minimum calculated pull-up (from section 7.1) you may need to break your bus up into multiple segments using repeaters, multiplexers, or something else. There are two advantages to measuring and then calculating, the first is that I feel it is quicker than trying to calculate your bus capacitance especially since many factors can affect it, the second is that you actually get to see the waveform and it can raise a red flag if the signal looks bad.