LED irradiance is a function of current, roughly speaking. LED illuminance is more like a logarithmic function of irradiance. So to achieve a roughly "linear" increase or decrease in perceived brightness, you will want the current in the LEDs to follow a rough approximation of the RC charging curve (which is exponential.) Unfortunately, just driving LEDs using an exponentially changing voltage doesn't do the trick. You may prefer to control the LED current as a function of the charging voltage.
The following circuit will achieve this to a reasonable approximation:

simulate this circuit – Schematic created using CircuitLab
The current mirror formed from \$Q_2\$ and \$Q_3\$ will operate reasonably well down into shallow saturation for \$Q_3\$. This means you can almost achieve the entire \$V_\text{CC}\$ voltage across your LEDs, less perhaps half a volt or so. And they will control the LED current the entire time, as well.
\$Q_1\$ is working as an emitter follower. As the capacitor voltage rises, so also does the emitter -- in relatively close lock-step. This means that the voltage on \$C_1\$ sets the current in \$R_\text{SET}\$, as the collector of \$Q_2\$ will always be only about a diode drop above ground.
The only trick in all this is that \$Q_1\$ requires a base recombination current to operate. This "drags" on the rising rate of \$C_1\$'s voltage and, likewise, accelerates the falling rate. However, this circuit uses only \$R_2\$ for charging but the sum of \$R_2+R_3\$ for discharging. The larger value of \$R_2+R_3\$ (which would otherwise appear to require a longer discharge time) is compensated by the recombination base current for \$Q_1\$, which also discharges \$C_1\$. So with a little bit of adjusting the ratio of these two resistors you can get approximately equal rise and fall times for the currents in the LEDs.
$$R_\text{SET}=\frac{V_\text{CC}-\frac{I_\text{LED}\cdot R_2}{\beta}-1.5\:\text{V}}{I_\text{LED}}$$
If you are using a \$12\:\text{V}\$ supply and want a peak of about \$20\:\text{mA}\$ in the LEDs, then using the above circuit you'd get something like \$R_\text{SET}\approx 390\:\Omega\$ (assuming \$Q_1\$'s \$\beta\approx 240\$.) Of course, it might be less than that too but this gets a ballpark resistor value to start with, regardless. (With only \$5\:\text{V}\$, \$R_\text{SET}\approx 39\:\Omega\$.)
Anyway, it's easy to try it out. So long as the LED current is modest (in the vicinity of \$20\:\text{mA}\$ or less) the dissipation in the three transistors should be within spec without the need for heat sinks. \$R_\text{SET}\$ should be at least \$\frac14\:\text{W}\$, though. Be sure to verify what I'm saying, by testing and feeling the change in temperature of all three BJTs and \$R_\text{SET}\$, though. Always verify and make adjustments where you feel better is needed.