There are two major components to make this work, the physical electrical interface and the logical data that is sent.
On the electrical side, the master by itself at least seems to be operating correctly if you see on a scope that the signals are good. Since you haven't provided plots, we can't tell if they are really good or whether you have a misconception what good is. On the slave side, yes, the pullups need to be weak enough to allow the slave to pull the SDA line low. If I remember right, the IIC spec says the slave only needs to sink 3 mA to bring the line below the maximum logic low level. If you are using 5V logic, for example, then this means the pullup can't be less than 5V / 3mA = 1.7 kΩ, so a 2 kΩ resistor would be a reasonable choice. Look at the specs for the internal pullup that you are using and make sure it can't source more than 3 mA. If it can, they you can't use it and have to use a external pullup.
On the logical side, you have to make sure the slave is being properly addressed. The IIC sequence should start with both lines high, then a start condition, which is SDA going low before SCL going low. After that for each bit SDA should change to the new value, then SCL go high, then SCL low again. The first 7 bits are the address, then the read/write bit. For the ninth bit, the master leaves SDA floating and the slave should pull it low soon after SCL goes low.
In my experience, the most common cause of not getting a response from the slave is incorrect address.