However, ESP8266 doesn't seem to share the same architecture, or does it? Is ESP8266 an AVR?
No, the ESP8266 is not AVR architecture.
Is ESP8266 an AVR? If not, how come we can use it as one through libs such as this one?
Generally the idea behind the Arduino project is to define a standard core API that will be the same/similar for any platform. The architecture-specific code is hidden behind this API. The "ESP8266 core for Arduino" that includes the library you mentioned contains this architecture specific code as well as the necessary toolchain. A separate hardware package is used for AVR, most commonly "Arduino AVR Boards", which is bundled with the Arduino IDE.
However, I think you're confused about the purpose of the ESP8266AVRISP library. It is used to turn the ESP8266 into an ISP programmer for AVRs. This does not require the programmer microcontroller to be an AVR but indeed the target microcontroller must be an AVR.