|
Look: Up in the Photoshop! It's Super Resolution!
Just when you want out of Adobe's licensing model, they pull you back in with an amazing new feature. This time, it's a machine learning AI-powered upscaling known as Super Resolution. And yes, it's also coming soon to a Lightroom near you.
Artificial intelligence and machine learning have become all the rage lately, but I'll admit it. I did not see this one coming. As you may know, Adobe Sensei, the company's branding for their AI platform, has been making appearances in several Adobe products in the past few releases. For our discussion here, you may be most familiar with it as the power behind "Enhance Details," Adobe's clever method of improving on traditional raw demosaic algorithms.
Digital cameras see in black and white only. Still, a system of colored filters arrayed atop the sensor grid records discreet green, red and blue pixel readings that can be combined later with neighboring values to synthesize a colored image. Standard methods work reasonably well when processing areas with even gradients but have increasing difficulty with fine details and high-contrast edges. If the next pixel over is supposed to be a completely different color, it doesn't help to factor both together for the final image. Doing so results in soft edges, false color, and other aberrations that would be best avoided. Think about the problem along the border of an image where neighbor data doesn't even exist on all sides.
Companies have been searching for a way to solve this problem for years now. Whoever can coax the most detail from a given raw capture wins the race. Usually, the edge goes to the camera manufacturer's raw converter since they have the hometown advantage. But the stakes are high, and the competition is fierce.
Back in 2019, Adobe upped the ante with "Enhance Details" in Photoshop Camera Raw and Lightroom. The feature employed machine learning in something called an "extensively trained convolutional neural network (CNN)." Never mind exactly what that means because what it means is improved resolution from raw images. While it did require a fair bit of compute power to perform this magic, it worked quite well if you had the horsepower. It also wouldn't work on older operating systems as it depended on code only present in Windows 10 build 1809 and Mac OS 10.13 or later. Also, note that since Enhance Details works via improved interpolation, they only work on raw captures. Even though you can edit jpeg images in ACR to some extent, these options are not applicable.
Over the couple of years since then, the engineers at Adobe have been hard at work coming up with even more ways to use their Sensei framework. Shipping with Camera Raw 13.2, something dubbed "Super Resolution" is now available. And it will also be released soon for both Lightroom and Lightroom Classic to make everyone happy. We all want higher resolution.
The first thing to be aware of with this new release is that Adobe has renamed "Enhance Details" as "Raw Details" since it depends directly on interpolating raw images. Super Resolution is similar in that it can be found in Camera Raw, not directly in Photoshop, but it will work on any image. If you open a raw file, Photoshop will automatically use Camera Raw. To open other image formats in ACR, you can do so from Photoshop or Adobe Bridge.
Super Resolution upsizes any image to quadruple the resolution at the touch of a button. Even if you start with a low-resolution jpeg from your phone, you can make it look almost like you shot it with a more expensive DSLR. The function uses machine learning AI to upscale images while preserving clean edges and detail without introducing artifacts.
Start with a one-megapixel image, and you'll end up with four. Run it on a twelve, and it will give you forty-eight. And yes, if you own a Nikon D850 like I do, its 45.7 megapixels will yield over 180 megapixels. Of course, you will need a reasonably beefy machine to handle such stratospheric heights of resolution. Computer companies keep making faster machines, and software companies keep coming up with ways to use all that power. That seems like an equitable arrangement to me.
Adobe developed the model by training it with millions of pairs of low and high-resolution images of the same small area. From this, it learned how to upscale the lower resolution ones to match the higher counterparts closely. They used a large sample covering a wide variety of subjects, particularly "complicated" ones, so the resulting code can handle almost anything you throw at it.
The Super Resolution output will be a new DNG file with a name based on the one you used. It does not directly alter your original. Although I haven't put such limits to the test, Adobe says the process is currently limited to images with a maximum size in any dimension of 65,000 pixels and a maximum of 500 megapixels in size. But fear not, they're working on letting you go even bigger.
Although the process will work on any image format supported by Camera Raw, Adobe recommends working from raw files for obvious reasons. You want as much starting data as possible, and interpolation to other image types is inherently lossy.
Regardless, make sure your computer's GPU is up to the task unless you like waiting. If you're in the market for a new computer, look for one with a GPU optimized for Windows ML or Apple CoreML machine learning technologies. You know Adobe won't be standing still. Although I can't imagine the details, you know they're hard at work further leveraging AI and machine learning.
AI and ML are the wave of the future. To coin a phrase: Up, up, and away!
| 
