I read somewhere not to connect battery or controller first (forgot which one was it), because I coded it myself and added some software power surge/failure protection, slow start etc. Do I need to care?
It is not particularly clear what this even means, but I am going to guess that the battery is the load (on the output side of the boost converter), or perhaps if you have separate power sources for the controller and the power in (what goes through the inductor).
Always always always make the power connections the last ones you make if there are separate power supplies for the control circuitry. Otherwise, you risk latching up the controller, or worse. The thing that controls needs to be up and running before the thing it is supposed to control is powered up. Otherwise, there will be nothing in control and that doesn't always result in a well-defined behavior.
Another often over-looked aspect is that if your load (rather than power source) is a battery, you should never connect it to the output before the boost converter is not just powered up, but has reached the intended output voltage on the output. Otherwise, the battery will just cause the ESP8266's internal protection diodes to turn on, which tends to make those diodes get superheated and vaporize, venting forcefully out of the chip's packaging.
This kills the ESP8266.
Some protection for micro controller is usually mentioned but I don't get what does it mean?
It means asking those little tiny CMOS digital IO pins to deal with something that is going to have 2 orders of magnitude more capacitance than any CMOS input which is what the IO pins are designed to drive. You're asking something meant to fill and empty a thimble to fill and empty a bathtub instead. The MOSFET gate will briefly behave like a dead short on that pin, and parasitics can cause voltage spikes on the pin as well.
The other thing is that your primary point of failure is going to be the MOSFET. Switching converters tend to cause a lot of ringing on the switching node, which results in voltage spikes that can sometimes far exceed your switching voltage. The oxide layer in MOSFETs is quite thin and fragile, and it is quite common for a voltage spike to not just punch a hole through from the source to drain, but also through the gate as well. Which will also apply that spike to the drive pin of your microcontroller, followed by whatever the voltage is that you were switching with that MOSFET. Unless you were switching 3.3V, this too will kill the ESP8266.
Do I need some discharge resistor for the MOSFET or the low signal discharges the MOSFET? (it is a logic level MOSFET).
The low level will discharge it, though it is a good idea to have a pull down resistor on the gate anyway. This ensures that the gate will stay turned off until something actually intends to turn it on. Otherwise, if the ESP8266's pin driving the gate goes high impedance for whatever reason, the MOSFET will probably just oscillate in response to whatever noise couples into the gate and that's not good if you have the power section
(the thing being switched by the MOSFET) connected.
I am using esp8266, using 40 kHz as frequency which appears to be max by docs, Any way to confirm if it is indeed generating 40 kHz without oscilloscope
Use a frequency counter I suppose. Or just disconnect the ESP8266 from the rest of the circuit and connect that output to another one of its inputs and use the ESP8266 to count/verify the output frequency itself. Or hook it up to a ultrasonic transducer (or maybe a piezo speaker) and see if it drives your cat or dog nuts. Both animals can hear 40kHz easily. Just don't do it for long, I imagine 40kHz would sound pretty annoying if we could hear it.
How important is input capacitor? I currently have none.
The primary goal of input capacitance on a boost converter is to reduce voltage ripple. This ripple will not only be on the input side, but will feed through to the output as well to some degree.
It can also influence transient response. However, assuming you have a separate power supply for your control circuitry, and are using a battery to power the rest, you don't strictly need any input capacitance (on a boost converter. Other topologies, like a buck converter, will not be as tolerant of this) because the battery could care less about voltage ripple. You will have added ripple on the output though. But is it absolutely required for the converter to function? Not really.
Keep in mind that bulk capacitors (electrolytic, tantalum) have too much ESL and resulting impedance to have much effect on voltage ripple. 40kHz is slow enough that they will do a little bit, but only if they are very very close to the MOSFET switch. Anything connected via the breadboard already has too much parasitic inductance between it and the MOSFET to do much good. Beyond that, bulk capacitance will not help at all when it comes to those edges of fast rise times. The only capacitors that generally can have a meaningful impact on ripple with such high harmonic content are ceramic capacitors. But only if they are essentially soldered directly across the pins of something like a TO-220 MOSFET. Even a few nanohenries of parasitic inductance will nullify most of their benefit, and the distance needed to reach nanohenries is on the scale of millimeters.
Honestly, with something so far from ideal like a breadboard and those long wires, adding any input capacitance would be ripple reduction theater more than anything else. The caps will look good sitting there but they won't actually do anything.
Output capacitor is 50v 1500 uF, is it good enough? Do I need bleed resistor? Do I need a ceramic capacitor in parallel to deal with the high frequency noise?
Bulk/electrolytic capacitors on the output are there to deal with load transients, not the output ripple. They are unable to behave like capacitors at all at most switching frequencies. They will have a small effect at 40kHz rather than none at all, but they will do nothing against any harmonic content. Ceramic capacitors are the only capacitors that really matter usually. Your converter is the exception, as the switching frequency is low enough that they can actually still behave like capacitors (albeit ones not nearly as large as they would appear to DC).
Lastly, if I build it on a PCB instead of breadboard, will it perform better?
I suppose, but any performance you might achieve is going to be severely limited by using a microcontroller as your control loop. Between the ADC having less usable bits than you probably believe it does, extremely limited bandwidth, huge amount of phase shift, quantization noise and relying on voltage-mode regulation, it is a losing proposition.
This is true for any voltage regulator - and a switch mode voltage regulator is actually just a linear voltage regulator whose output sets the duty cycle of some PWM generator.
The only reasons to use a microcontroller for this purpose is for educational purposes, or in extremely cost-optimized designs where a microcontroller is already present, and one simply needs to generate a higher voltage at low power feeding an easy/fixed load and without much concern for efficiency or EMI. Frankly, it would be a waste of effort to create a PCB for this controller unless you choose a more practical controller.
You need an analog feedback loop that uses current-mode rather than voltage-mode regulation, and not software control. I'd suggest looking into PWM controller chips - think of them as 555s on steroids - this is what most commercial SMPSes used to be built with. Chips like the TL494 or SG3525. The nice thing about these chips is you can use them to do all sorts of other interesting things too, even build an isolated converter and other interesting things. Even these are kind of old-school now, most new designs use more application specific buck/boost/etc controllers. They can still be useful when you need lots of watts (hundreds) though.
But you will simply not be able to achieve good regulation using the ESP8266, so the boost converter will work as long as your load is well behaved and your boost ratio small enough and your MOSFET doesn't get too hot. But I think you will find yourself getting frustrated very quickly as you run up against the limitations of using a microcontroller as a switching regulator. It is however great for education and learning how switchers work and behave, at least to some degree.