The R2FX may consume less than 1W (50mA), but the Raspberry Pi 2 B can use up to 2A or 10W. Granted, it should not consume this much constantly.
What might work well is two of the 3.8v cells in series, giving 7.6v. Or even better, three of them at 11.4v. Then use a buck converter such as the LM2670-5.0 to switch and regulate this down to 5V. The reason why is, a buck converter will draw less current from a higher-voltage supply, extending the battery life. Boost converters draw more current and shorten the battery life. More current always means more losses. So even if the three batteries were placed in parallel (3.8v but 3x the Ampere-hour rating), there will be more intrinsic losses in boosting this to 5v. (However this is easy to recharge. Something like the BQ2054 or a hundred others could work.)
Yes, lithium batteries require special charging circuitry. If not used properly, they can be quite dangerous.
The down-side to using a buck regulator is the charging of the batteries. For simplicity, they could be charged individually using their old chargers. To charge them assembled in series is more complicated. Often they are just treated as one battery, however imbalance could occur, shortening the life of one or more cells. This is especially true for lithium-iron-phosphate types (probably not used in phones.) You could try researching "lithium charge balancer" or similar.