Stop thinking Arduino and challenge yourself learning the bare metal microcontroller. 4k flash is not a lot for an Arduino sketch and none of the Arduino libraries are guaranteed to work, although some do. It is not supported though and there is no definitive list of libraries or library calls that work on different microcontrollers.
Check the ATtiny4313 datasheet and search for TWI (two wire interface). The datasheets is like the bible for your microcontroller, everything you ever wanted to know about it is written in there and in my opinion the Atmel datasheets are pretty good to read. From experience I know I2C is a bit tricky to get working, but you can check the Arduino library for how they solved it. The ATtiny and ATmega families are closely related and the code will be reasonably easy to port.
Acutally what you can do is work from your Arduino and port the Arduino specific code line by line until all Arduino specific commands are ported. Easy to test, and you can do it step by step. Then porting from ATmega to ATtiny is pretty straightforward.