I need to develop an Ethernet encryption scheme. All that needs to be done is take the input on Ethernet and encrypt the packets and send them out the output Ethernet. Is there ANY way I could do this for $35 a piece? How about $50?
I have a lot of concerns with this project, so let me brutally honest here. When it comes to cryptography the most dangerous thing is for someone who doesn't know about it to try implementing it. And honestly, you don't know about it. I say that because of your lack of caring about what encryption algorithm is used. I also don't think that you really know the computational resources required for public/private key encryption.
Even if you chose the correct encryption algorithm, and implemented it perfectly, that doesn't mean your device is secure. Most encryption security breaches are not from someone doing a brute force attack on the key, but from something else that is much easier and has nothing to do with the algorithm used. For example, if you encrypt the ethernet packet headers then you'll have better security, but you won't be able to send this data through a standard ethernet switch and/or layer 3 router. If you don't encrypt the headers then you're opening yourself up to a form of analysis and attack that could give someone information even without breaking the encryption.
A book that I highly recommend is Applied Cryptography. It talks about lots of issues with cryptography, and also goes through a lots of algorithms. It is a little dated in that it doesn't cover AES, but it does talk about many hash algorithms, DES, and public/private key encryption.
Ok, now on to your real question: Can it be done for $35/each? Absolutely! Can YOU do it for $35/each? Well, I can't answer that. You simply didn't give us enough information. For starters, is this a hobby project or something that will be made in volume? What is your case/chassis going to be? How is this going to be controlled, via LCD and buttons?
If you are only building two of these things then it is going to be very hard to get the price down to $35. It would actually be hard to get the price down below $250. But even in volumes of around 2K/year I would estimate that the PCB+parts would be in the $30-35 range and that doesn't factor in the cost of labor, a chassis, or an AC/DC power supply. Also, there might be cheaper ways to do this that don't use an FPGA.
The lowest cost hardware with two ethernet ports and some processing power between would be two PIC 18 67J60 connected together. Each has it's own ethernet interface, and they could communicate using their built in UARTs between them on the same board. The parts cost should be under $20 for the two PICs, two transformers, two RJ-45 jacks, and the board. If you already have regfulated 3.3 V available it needn't be much more than that.
That setup is certainly capable of passing packets from one port to the other and applying some sort of modification in between. Whether is is fast enough for what you have in mind we don't know. That all depends on how complicated your encryption algorithm is and what data thruput you expect.
You say you are only dealing with packets at the ethernet level. If that is really true, then something like this might work. Without a full blown TCP stack, the chips should be able to handle a higher data rate. However, have you really thought this thru? How do you know which packets to encrypt without at lest some analysis at a higher level? You could transform only TCP/IP packets, for example. That's easy enough to detect, but do you really want to scramble the whole packet, including the TCP header? What about the IP header? If you scramble that, how are your packets going to get routed? It doesn't sound like you've really thought this out all the way. Hardware to do this is probably the least of your problems.
How about a PC with two network cards?