Page in Construction. Feel free to fix any grammatical or spelling mistakes or add analysis on subjects other than the ones considered below.
The most ancient event mentioned in the series is the creation of the D-Reaper program in the 1970's, which in the real world did happen but it was simply called Reaper and was only used to delete one of the very first virus, the Creeper. As such, the program had to able to distinguish between a normal program and a virus (even if in a rudimentary process). While in the real world nothing else came of that, in the world of Digimon Tamers it was also able to integrate the useful abilities of the deleted programs and therefore remained idle in the dimension based on the ArpaNet, not a early digital world. On the other hand, the Digignomes were created in the early 80's, at a time of booming research in artificial intelligence, presumably representing part of the so called "expert systems" (one of the very first AI's) with the part that contained knowledge being represented by the library shown in episode 32.
The next most important event was the formation of the wild bunch/monster makers, who devoted themselves to the development of artificial intelligence in the late 80's. While their efforts would ultimately result in the creation of the digimon, they were far from being the only ones. This second period of booming research (which led to the development of the world wide web) was what triggered the awakening of the D-Reaper onto the digital world, in other words giving it the ability to cross between different dimensions for the first time.
Once in the digital world, it started deleting most of the projects of that time including the Tierra Project. For the digimon's perspective it was like an apocalyptic event which led to a mass extinction but, as you recall, the D-Reaper was able to analyse the programs before determining whether or not they should be deleted. Considering that all these projects of artificial life all obey the same basic principles, the D-Reaper concluded that all life is by definition a competition for resources without any productive goal.
As the D-Reaper returned to it's original dimension, the Wild Bunch/Monster Makers dissolved due to lack of interest, but Shibumi continued working on an evolutionary algorithm with helped the digimon grow that much faster to a greater strength but most importantly, intelligence. Meanwhile, the SINGIT systems of which the first was Echelon have gained importance with the gradual expansion of the network, leading the Japanese government to create it's own counterpart, Hypnos, and naturally gave it far more capabilities in order to gain a technological advantage.
It's also heavily implied that it were these superior capabilities (especially the ability to interfere with the network itself) coupled with the high density of population (meaning more information produced) that led to the breach between the digital world and the human world in Japan, explaining why digimon don't realise nowhere else. On the other hand explosive increase of the number of internet users on the turn of the millenium grew as never before (due to easy to use OS's like windows 98/2000 and the spread of broadband... in Japan), "feeding" digimon and making their development faster than the growth of the network, thus leading to the awakening of the D-Reaper i.e., the start of the series.
The explanation given in the show is that when crossing between worlds, the digimon's data form the corresponding proteic structures. Though, based on the imagery used in the series, I believe a more detailed explanation can be given:
Looking at the card slash sequences, we can see that the slashed card is split on 3 planes: the digital, the analog (represented by the graphs) and the real world. The series shows the digital world and the real world, but for brief moments (like in the very first episode) it also shows the analog world. This world is more important that it seems, because it's there where digimon's raw materials were collected from the real world's atmosphere, since they had to come from somewhere and air does contain all the basic elements necessary for life like carbon and oxygen (CO₂), hydrogen (H₂O), nitrogen, and others in trace amounts. The reason these elements are collected is because this world is essentially composed of energy, to a level that ionizes the air. This would explain the behavior of the fog that sets in when a digimon realizes, namely it not dispersing (since it's particles are attracted to the digimon) and it dissipating without a trace over time after having discharged (which happens faster the bigger or the more digimon there are) or after the digimon that attracted it is killed. Also, keep in mind that digimon do not form instantly, and thus the digimon that just realized is the one that is most unstable/possesses the most charge. Based on the above, the realizing/bio-emerging process can be described as: digimon is composed of data particles -> these data particles induce currents on the analog world -> these charges trigger motion of the charged particles, transferring potential energy  -> all the potential energy that was in the digital world is consumed in the end.
While going from the real world to the digital world is inherently safe (since if a molecular bond is broken there is always a defined energy transfer, i.e data), the inverse is not true because the whole process is not 100% efficient (due to thermodynamics) and the energy taken from the data construct may not be enough to form molecular bonds. This makes world crossing an exhausting and dangerous process (which is why only the reasonably strong can cross over), and is also the reason why a arc was so desperately needed to bring the Tamers back to the real world. Yamaki was certainly not going to risk the kids' life, and note that he was very much aware of this risk because it was through the analog world that hypnos destroyed digimon. After all, that white void, the original Yuggoth program and Shaggai/Juggernaut (ver 1 and 1.5) all operated between the real and the digital world (ergo on the analog world). Shaggai/Juggernaut was even specifically designed to take advantage of this process! Remember that after forcibly attracting the digimon to the analog world, it created a huge magnetic/electric/electromagnetic/whatever field intense enough to affect the digital world (creating lightning storms). Most importantly, that field scrambled the conversion process, so instead of a molecule directly inducing data constructs, that obtained potential energy was forcibly dispersed, resulting in the digimon literally felling their body being ripped apart.
First and foremost, it must be said that in this series evolution is directly connected to culumon/calumon. As you may recall, Culumon was described in-series as a digital entelecheia, and Konaka's character notes confirm that it was a reference to the original Aristotelian concept. In order words, Culumon is literally potential given form. In the real world, this is as strange as if all forms of light itself became a single physical sphere (ergo, nobody would be able to see a thing), but since we are talking about a digital world (in which the rules of physics are a simulation) it means that the world itself can theoretically be warped. And that's exactly what the digi-gnomes did: they rewrote the digital world (but not the D-Reaper, since it resided in another world) and inscribed the program that allowed shapeshifting/evolution on a white cutie (i.e, the digimon were locked in their forms).
Note that in a way, they made the digital world more similar to the real world (well, do you happen to see things increasing their size and changing their shape instantly?). Yet, digimon do evolve in the real world, and they achieve this by temporarily crossing to the analog world (the space between the real and digital worlds), where as energy beings they can change shape freely. Or could if that world allowed it but as it is they need Culumon to allow them to change shape in the digital world (which is why he's described as the catalyst of evolution.) Afterwards, their new shape materialises in the real world thorough the realizing/bio-emerging process described above. Note that the process is much safer that simply crossing over because the process can only start after the energy requirements are met (and fortunately they never tried it when Shaggai/Juggernaut was active...).
The meaning behind the evolution levels on themselves is more uncertain, but based on their original japanese names (child, adult, perfect, ultimate) I would guess that the standard evolution is simply going from child to adult (or from adult to perfect, or from perfect to ultimate), and because it's a matter of development it could be achieved either by age (which is why Babamon and Jijimon are Ultimate/Mega level. Seriously) or by taking experience from others which literally means eating their remains, since they are data constructs in the world of Digimon Tamers. Of course, what the digimon didn't know is that they were ModeLocked, so they kept trying harder and harder to become stronger in vain. They even tried to absorb data in the real world which although it is possible since digimon can act as electrostatic(?) particle attractors, didn't help them any.
Higher evolution on the other hand takes even longer unless you have a certain evolutionary algorithm. Keep in mind that (in the original japanese version) matrix evolution applied to evolution skipping a level to perfect and and two levels two ultimate. Why? Because in both cases there is a specific interaction between two pieces of data. Considering that the digimon have to be able to materialise in the real world, it's highly probable that most of their data is in matricial form, since they are used in physics, with a certain amount of variables. Now, since Shibumi was part of the digimon project, he certainly had a good idea of how the digimon's data is structured so his program was designed to interact with the digimon's data, perfecting it. On the other hand, the kids themselves can also be defined with a bunch of matrices, except being humans they are far more advanced than anything a programmer may have created thus combining the data of the two  will produce a digimon far beyond the original projections, which is advanced as it can be (note that I'm not saying they can't learn or become more powerful), in other words they are on the ultimate level they can achieve.
The initial stages involved loading a version of the Shaggai program into Terriermon's ears, when in the form of Saint Galgomon would provide the centripetal force needed to produce a reversal spin within the wormhole between the Real World and the Digital World created by the D-Reaper. Yamaki then activated the Shaggai program, causing the direction the transphotic eddy to be reversed. When the angular velocity exceeded the speed of light, a backward flow of time was created, with the “Big Bang” effect produced by Shaggai having deminished the D-Reaper Zone it gradually began to dissipate through the vortex.
The reverse flow of time devolved the D-Reaper back into it’s original, harmless state but coming at a cost. The Tamers' digimon began to devolve themseleves like the D-Reaper and had to return to the Digital World to prevent from completely disappearing from the plane of the Real World.
- Basically, it was a straightforward usage of real-life programs like Destributed.Net, SETI home, etc. The idea behind them is simple: Install a program on a random computer connected to the internet. Afterwards, that program tells Hypnos the computer's address and capabilities. Hypnos then sends a proportional amount of raw data through the internet along with the necessary algorithms and instructions (how the data should be processed). After completing the calculations, the program sends back the results and waits for a new batch of data. This process only paysoff if the amount of data to process is very large, since the task can be split onto a very large number of computers. The program itself was sent as an attachment by an e-mail to a large number of households, and said e-mail wasn't seen as Spam because the senders' address was that of the Japanese Goverment. Of course, there are always those that didn't recognise/trust the message anyway, but since it was sent to at least all of Tokyo, if not Japan, they still got to more than enough people in record time. On the under hand, this action did go a long way towards revealing the existence of Hypnos to the public which meant the general population was now well aware that the government could spy on them whenever they wanted. Not something that's easy to explain.
- Although no one has actually built one to a usable scale, a quantum computer by nature can handle large amounts of data more easily, thus explaining it's very quick adaptability. On the other hand, it needs the increased processing power because one of it's objectives is to analise subjects to determine whether or not something should be deleted. The reason why the Wild Buch/Monster Makers had been able to guess this beforehand was because the D-Reaper had shown them that it could analyse and then interpret (thus revealing that the mega level was a fusion) matter on a molecular level in real time, an incredible feat far beyond any technology available even today. It also opens the possibility for Quantum entaglement, that is to say faster than light transmission of information (since in a quantum computer a qu-bit is represented by the state of a particle) which does become very useful when the D-Reaper needs to coordinate itself on a planetary scale. On the other hand, sheer processing power doesn't mean anything if the software is unable to process the data, and the series shows this when the D-Reaper struggles to assign reason to the act of parenting . This is due to it being a cultural concept which is clearly important to Jeri's mind, but since it is beyond the scope of the individual it cannot be understood by a program that is used to dealing with other programs whose purpose is defined in themselves, ergo, just have a self-contained purpose (even if that purpose constantly changes in the case of AI's).
- okay, I'm not being very accurate but the explanation is understandable anyway and I'm trying to keep it as simple as I can
- In the original dub, Yamaki had remarked that it was a bug in the Hypnos system
- imagine the process as a moving ball (ion) hitting another that is still (data). Provided the elasticity is high, the ion ball will stop (representing dematerialization) and the data ball will start moving slightly slower, thus having a successful conversion. What hypnos did was hit that data ball so it stops/go where it's not supposed to, thus preventing conversion
- that the kids can be completely defined in terms of data is proven by them being able to go in the digital world without losing any mental faculties (like free will). Do not read any further than this
- I would also like to point out that Hypnos didn't hack the signal and then showed it to the rest of Japan (which would have been strange, to say the least). It was the D-Reaper that Overwhelmed the Televison signal in a display of force