Striving for Real-Time Photorealism in Architectural Visualisation
Lasse Rode takes Unreal Engine for a good spin as he strives for photorealism using a real-time tool. He does this with his great looking, and mostly white, Berlin Flat scene as seen on the forums. Follow along as Lasse explains getting started with Unreal Engine 4. Enjoy it!
Author: Lasse Rode / Studio XOIO
The Berlin based studio xoio has been established in 2006 by Peter Stulz and Lasse Rode. It is specialized in high-quality emotive architecture and product visualization. The team at xoio consists of people who have an architecture and design background. That provides a strong understanding of creative processes and the ability to give support in questions of design.
Introduction
My name is Lasse Rode and I am part of studio xoio. We are a small agency specializing in visualization and illustration works for architecture and product marketing. Usually we work in a kind of “traditional” 3d environment utilizing applications like 3ds Max and the like. We are constantly checking out new rendering engines and currently making big use of Corona Renderer, V-Ray and Octane. Each engine has its strengths and we always try to use it like that: each for the best purpose.
In my eyes there have been several main trends within our industry: The strive for (photo)realism and becoming faster – if not real-time. Not having to wait for your rendering coming out of your render farm was always a dream for us – especially when rendering animations!
The main downside on the “real-time-thing” for a long time was the lack of quality you could achieve with it compared to pre-rendered still images or animations. So even though it looked very interesting, the application of it in a visualization context seemed to be hard to imagine – and honestly, the “gamey” look of it made it hard to sell for high-demanding clients from the architecture and brand fields.
This has changed rapidly. The results possible with real-time engines today are very beautiful and convincing!
Why Unreal Engine?
The release of UE4 gained a lot attention within our industry and the possibilities seemed to be endless. The PBR (Physically Based Rendering) material system and the easy-to-use importing pipeline for getting your models into the Unreal Engine was the most compelling reason for us to give it a try – in addition to the quality of output possible! If you have seen the work of Koola (also available as a download in the Unreal Engine Marketplace) which went viral some weeks ago – you are probably as convinced as we are that Unreal Engine 4 is capable of impressive quality.
In the following article I want to give you an outline of the workflow I used to make the Berlin Flat scene you can download from the Marketplace and share some techniques and tricks I came across during the process. Some of them I found myself, while others are derived from information I found on the web. The Unreal Engine Forums and the Documentation are a vast and great resource, as are the starter content that comes with the engine and the assets and scenes you can get from the Marketplace.
The Berlin Flat
I made a series of images of this flat in a historic building in Berlin beginning of 2013 using 3ds Max with the Corona Renderer. It’s flexible way of handling the color-mapping, which really helped to pull off the very whitish mood of the whole set. This actually was also the reason for choosing it when giving UE4 a try.
I noticed UE4 being very successfully used on scenes with gloomy lighting and busy textures. I suspected it to be not that easy to get precise shadow and GI within an ultra-white interior.
And honestly: It is a tricky task!
Above is one of the original renders done with 3ds Max and Corona Renderer. To have a look at the entire set, click here. Below is the UE4 video made with this scene…
I will start from the beginning :
The Original Scene
The entire modelling was done in 3dsmax in a rush, so I actually detailed only the parts that are visible in the final images. This is an approach that of course is not possible in a real-time environment. Re-purposing the scene for use with Unreal Engine I had to reduce the scope a bit because furnishing and detailing the complete space would have taken too much time for testing purposes.
I decided to export only two rooms : the ones you see on the lower part of the screenshot above.
Exporting the Geometry for Unreal Engine
This is a very easy task IF you keep some things in mind!
It makes sense to split things up a bit. Because the lightmass is calculated into a separate map for every object, it is good to be a bit careful with high values especially on big plain objects like walls and ceiling. Because of this I only exported the inner faces of the walls which we actually see.
I also added a bit to the top and bottom of the walls to intersect them later with the ceilings. I found this to be a good way to prevent “light-leaks” – lighting artifacts that happen when geometry is not closed or not intersecting. This is no problem when having a gloomy scene with lots of busy textures – but, since we are going to have an ultra-white space it is important to get as precise GI as possible, especially in the corners.
The second crucial thing is to create unwrapped UV coordinated for the channel the GI is going to be stored in by Unreal Engine’s lightmass calculation. In 3ds Max this would be UV-channel 2.
Channel 1 is for use by all the other textures like diffuse, roughness, normal, etc. Unreal Engine counts the channels starting from 0, which can cause some confusion in the beginning – but once you get it, it is fairly simple.
Note : Unwrapping is only important for the light-map channel! For the texture channel any kind of mapping can work, such as cubic or cylindrical mapping. In most cases a simple “flatten mapping” in 3ds Max unwrap does the job to create sufficient UV-coordinates!
If you want to put your scene together in UE4 like it has been in your max-scene, it is good for entire “space” to leave it in place when exporting because the object’s coordinates are easier to align. For single objects like chairs and other assets it is very comfortable to export it only once and instance them in your Unreal Engine scene. For this purpose it is good to move them near the center of your 3ds Max scene because the new object’s pivot in Unreal Engine will be there.
You see I am using high-poly geometry without any LOD (Level Of Detail) simplification. This of course is only recommended in small scenes like this one but since I’m after a smooth experience and don’t want to have any jagged edges on my furniture, this was logical for me. I have no doubt there’s room for optimization, though ;).
Make sure your assets are merged into one object and have different material-ID’s applied to handle the different materials later in UE4!
Well, then you save your geometry as .fbx and off you go over to the Unreal Engine editor!
Importing into Unreal Engine 4
Importing FBX files into Unreal Engine 4 works pretty smooth! I did it in several steps.
I prepared different files that made sense :
- The geometry of the room in a separate fbx file.
- Different file for the assets, each with some objects in them.
Just make sure to uncheck the “Combine Meshes” to receive your objects separately and not baked into a single Mesh!
Materials
I’m a very straightforward guy and a big fan of simple setups! It’s kind of a philosophical thing, but achieving things with the least effort possible is far superior to using a setup only you understand or you can’t remember when opening a scene half a year later.
So this example of a shader is very simple, consisting of a diffuse map, desaturated and blended with black color. The same map then is color corrected and inverted to put into the roughness channel of the material. Done.
A normal map would have been too much here, but feel free to explore the materials for yourself in the scene!
Here you see the wood material applied to the chairs and the table – a dark dyed wood with a crisp matte reflection revealing the wood structure and texture.
In this image you see two more materials which might be of interest, the curtain which is back-lit by sunlight – a two sided material.
You have to set the Shading Model to “Subsurface” and add a constant node with a value smaller than 1 and wire it to the Opacity property of your material to get this effect.
The jar in the foreground has a very simple glass material applied :
It has a fairly dark diffuse color, zero roughness and a high specular value. I also involved a Fresnel-node with a value of 1.5 to control the opacity and refraction. There are a lot more complex ways to generate more realistic glass – but I honestly had some trouble to really get control over that, so this easy glass seems to be good enough as well 🙂
Note I checked “Two Sided” and set Translucency Lighting Mode to “TLM Surface” in the Details tab on the left.
One material I want to show here as well is the floor, because this one is the only one having a normal map applied.
Here you see a material defined by a diffuse color, a roughness texture and a normal map. The diffuse color is a simple very light grey, defined by a 4-value constant.
Roughness looks a bit more complex: On the left you see the same map three times scaled differently with a TexCoord node. The red channel of each then is multiplied with the others and then wired as an alpha into a Linear interpolation node (Lerp) to blend to values. 0.3 and 0.2 in this example to get a subtle noisy reflection on the floor planks. This then is fine-tuned with a “Power” node to get just the right amount of roughness that looks OK.
The normal map again is influenced by a TexCoord and then being flattened by a fair amount via a “FlattenNormal” node to get just a subtle relief on the material.
Preparing the Assets
Before dropping the assets into your scene it is always best to apply the materials onto them within the geometry editor. You would have to do it only once and still can apply different materials in the main scene if needed. This is a fast process: Here you see it is important to apply different Material-IDs to your objects to put the different materials where they belong!
Building the scene
Well, this is kind of brief, but: put the thing together. First you have to drag in the room geometry. Best way is to select all the parts needed and drag & drop them into the empty scene. Afterwards all the furniture and assets have to be placed in the environment.
Here you (don’t) see the back faces of the outer walls – as I’ve explained above: they are only single-sided for better lightmass calculation.
For this exact purpose it is also good to set the lightmap resolution for your larger objects to a high value, for the walls I set at 2048 for example.
As mentioned above, light leaks can be an issue. To prevent these I put black boxes around the whole scene. It looks kind of messy from the outside – though more clean on the inside 😉
Lighting and Lightmass
The lighting also is a fairly simple setup : I used the “Koola method” – a combination of a sun and planes with spotlights in front of the window to simulate a skylight. It is rather effective and easy to control!
To now calculate the global illumination only a few tweaks are important :
I drastically increased the lighting bounces and the indirect lighting quality. I also decreased the smoothness to 0.6. Details are pronounced better and the shadows don’t wash away so much.
I also set the direct lighting to a dynamic shadow for getting better shadows. This also is important to have the light moving later in the animation!
Last step before hitting “Build” would be to set the Lighting Quality to “Production”!
This should result in a smooth lighting everywhere!
Actually when getting to this point the first time I was kind of thrilled! This is actually the strongest part of this engine: to thrill you! Being able to move inside my “rendering” in real-time was really a delighting moment!
“Post Processing”
One of the greatest features is the possibility to apply color correction and camera effects just within the editor. This can be done with a PostProcessVolume for global settings. I did some tweaks on the saturation, fringing and vignette, the bloom and disabled the auto exposure by setting the min and max values to 1 and increased the overall brightness by setting the Exposure Bias to some 1.42. I also added a lens flare which I find really awesome happening in real time!
Setting up the Animation
The ability to move freely inside the scene makes doing animation a very easy and pleasing task because of the instant feedback nature of the real-time environment. As a frequent user of compositing software it took not much time for me to adapt to the integrated Matinee tool and set up an animation.
First thing to do is setting up a Matinee Actor.
When opening Matinee you will see a window with a track section and a curve editor.
Setting up the cameras and animation work is very self-explanatory. Motion is controlled by key frames and curves just like any other animation software. Also the “cutting” work is done just within the Matinee editor.
I created a couple cameras moving slowly through the space: seeing exactly what you are doing really helps to tweak the timing of cuts and speed of camera movement!
You can see the camera trajectories just in the editor and control the editing on the fly!
After getting the rough “cut” done in Matinee, I then exported the complete animation as an .AVI and fine-tuned it in Premiere and aligned it to the music.
Conclusion
The entire process starting at exporting from 3ds Max and importing into Unreal Engine 4, working out the shading and lighting to produce the animation and then posting on YouTube took me about one day. This speed is unheard of in ArchVIZ and reflects very much the key potential that lies in the use of Unreal Engine 4 for visualization works.
The absence of render times in terms of “producing” images really makes the process of creation very flexible and free. The fast feedback of your action is the real revolution!
We are constantly thinking and testing the possibilities of applying this kind of production and workflow into our daily work and our environment as a whole.
There are a lot of possible applications and we are very eager to explore them!
I hope I gave some insight into the my motivation and process and wish a lot of fun with the Berlin flat scene 😉
Lasse,
xoio
ronenbekerman Amazing – thanks!
Great article! Thank you Ronen for bringing this issue…
MilosStojiljkovic And bit thanks to Lasse_R too! 😉
ronenbekerman MilosStojiljkovic Lasse_R you’re very welcome 😉
lasse1309 ronenbekerman And Lasse Too 🙂
I’ve been transitioning to Unreal Engine 4 for some projects and I can say this is a very thorough walk-through of the process! Giving a presentation on it tomorrow in fact. I would just add that for some scenes where GI is not as necessary you can go fully dynamic with your lighting and run the regular AO with decent results. Advantage is no bake time. So you need to move an object or change a light? No problem, move it and hit play.
Also, though not mentioned here, blueprints are your friend. They are the programmatic end of Unreal and really open up the door to huge potential for interactive real-time archiviz. Throw an Oculus Rift on there, program some design options or what have you that are proximity sensitive and clients won’t know what hit them!
Thanks to Ronen for his attention to these up and coming technologies!
samuelconlogue thanks 😉 well, you are very right with what you are saying! the full dynamic thing works quite good in some cases, especially with exterior scenes! and yes, the blueprints are not covered at all 🙂 did not touch them though for this test but have made great experience with it in meantime: huge potential, really!
You don’t really need uv 2 mapping any more with automatic lightmapping, wich is included in ue 4.5 and 4.6
Amazing ad very informative, Lasse.
For others like me, who have never even opened unreal, could you explain the steps regarding “create unwrapped UV coordinated for the channel the GI is going to be stored in by Unreal Engine’s lightmass calculation”.
2kemon as I said, you don’t even need that any more, unreal will do it for you kindly 🙂
2kemon well – unreal will calculate some kind of GI – which is called “lightmass”. briefly explaned it is similar to a irradiance map. this has to be stored somewhere. in off-line renderers like vray this would be the RAM and from there it would influence the raytracing in the way we all know it.
in unreal engine this is stored in bitmaps which are referenced onto the objects via a UV channel – and because every spot on your geometry is a different spot in your GI solution this UV coordinate has to be “unique”, which means it has to be unwrapped. this can be done in max beforehand – or automatically by unreal engine. when usin the max unwrapper you naturally have more influence on that. in many cases a “flatten mapping” would do the job – so it is quite fast process as well 😉
cheers
L
Great post, is there any chance we could get hold of the scene or a severely cut down scene? Would love to test it on my machine to see how it ran.
maxfury the scene is free for download in the unreal marketplace. once you registered, you can easily downoad it 😉
Ekanaut yyoooink
BryanRenno good shit, right?
Ekanaut It’s pretty impressive to see a lot of archviz work in the engine. I’d like to dig into their workflow and learn
BryanRenno That post is making me want to dive back into Unreal again 🙂 I’ve been in Unity land for awhile.
Ekanaut Both are great for different reasons. But UE’s visual pipeline is pretty fantastic
Demruth UE4 is amazing. Hopefully I’ll be able to play around with it some day.
Thanks for the great write-up Lasse. Been looking at UE4 since Koola’s first posts on the topic, and I think it’s definitely a promising tool for real time arch viz. Based on tests of my own, scenes shared by you and Koola and other resources out there, I think the only problem it currently has is the quality of the lighting, especially GI (lightmass). Speaking in terms of visual quality, I think the lightmass just doesn’t cut it when compared to GI solutions of proper ray tracers like V-Ray, Octane, Corona etc. I think it’s quite evident when comparing your still shots and the UE4 animation of the Berlin flat. The UE4 results look good but lack the subtle detail and tonal variations provided by a more robust GI solution. It only makes sense considering the purpose of UE, but still I see this as a deal breaker for high end arch viz production.
I should also say that by analyzing Koola’s work, you can see that he in fact gets away with the lack of GI detail by using dramatic materials (lots of glossy reflections, lots of detail on diffuse / reflection maps) and by having heavy contrasts in scene lighting / brightness. A predominantly dark interior with bright lights and lots of reflection is not as critical to the lack of GI detail as the white interior of the Berlin flat, which in my opinion has to rely more on high quality GI lighting for visual interest. From my experience with UE4, I believe materials are in fact one of its main strengths, you can get results that are very close to ray tracers on that front. It’s only the lighting that doesn’t really cut it in my opinion.
I’d be very curious to hear from you and other people who are familiar with UE4’s lightmass, especially in terms of the accuracy it can provide, and if you’ve find ways to boost the visual quality beyond the “best” settings provided. If there would be some improvements to it, I could see UE gaining massive interest in the arch viz scene.
P.S. it’ll be interesting to see what Unity 5 brings to the table, they are promising a real time GI solution that also looks pretty promising, at least on their tech demos. Interesting times indeed!
maxdaiber 2kemon thank you Max! So if I build a complete scene in my main 3D modelling app, texture it and export it to UE4. Can I then in UE4 add more detail to my basic texturing (bump, normal, refraction maps etc?)
2kemon maxdaiber Basically yes. If you work on max for example, you have to use standard materials, and you can also add normal maps, but every other map needs to be added directly on ue4.
Once working inside unreal engine you will have tons of information in the unreal engine website
Ville Riikonen hey Ville!
thanks for your long comment;
well – you are right: the lighting quality of course is different to the ray tracers we know – and you really have to push it hard to get it there!
honestly: i don’t really see the “real time” thing replacing still renderings in any way – but you also have to consider that rendering animations the traditional way also involves a lot of optimizing – sometimes resulting in poorer GI and material quality than you would have in a still image.
the use of busy material – like koola did – helps a lot of course to cover up strange GI artifacts. the main reason for problems on that front lies in the lightmass/baking pipeline which only allows a certain resolution per object, cranking up here would result in incedibly long baking times unfortunately. I wounder how things would look if UE would use an irradiance map like we know from vray?
but for sure real “real-time” GI would be better, but here’s also the qustion of resolution! right now theres the real time shadow casting for parallel lights with the shadow cascades. these also depend on some kind of spacial resolution; and you actually need that because compared to traditional still or animation rendering you cannot work with the pixel-space of your image-plane due to the fact we are not only real-TIME but also real-SPACE –
real 3d! not the faked one we are used to 😛
this is not the end yet, for sure 🙂
kind regards
L
lasse1309 Good point on the animation optimization. In fact you don’t need as much quality to animations as you need for stills when it comes down to GI for example. On a high-end still image everything has to be there, whereas on animations some areas and features don’t need that much detail or quality as the viewer won’t really be looking at them anyway. So in that sense UE4 and other real time applications for animation purposes are very promising! Ray-tracers for still rendering they won’t replace (at least anytime soon), I’m with you on that.
Haven’t looked too closely into the theory behind the lightmass calculation, but I do feel like it could benefit from a more robust / accurate GI algorithm, maybe something like irradiance map. The guys at Epic have probably optimized it for great speed on accuracy levels typical to what people use in games, where honestly speaking GI quality isn’t usually that important – it’s more about the impact of direct light, dynamic shadows, explosions and other effects etc. Nobody is interested in studying how indirect light interacts with the corners of a room, the color it catches when meeting a colored surface and all that other stuff us arch viz guys are all about 😉
It’s true what you say about the spatial calculation / 3d-grid of GI lighting. Again, I’m not the math guy but it’s probably a whole different animal compared to what ray-tracers are dealing with, and trying to push the quality up to the same standards is no small feat 🙂 Once we do get there (or quite close, at least), I think the production of animations in arch viz will change dramatically, and for the better I believe. Waiting for rendering just feels like such an unnecessary step in the pipeline, also in the sense that it hinders our ability to push the art in terms of iterating the work, testing how things look and behave while in motion and so on.
Very interesting times indeed 🙂 Best regards,
I tried to make a realistic animation of a new building complex, but it is still difficult to make a real-time animation with unity3d. Here you can see the animation: https://www.youtube.com/watch?v=BdnpyUxLNs0
You can see the major improvements unity 5 has to offer!
2kemon UE4 have one channel for textures and another channel for baked light.