FADGI in Practice

Refining Toward Better Performance

As we learned the practical implications of measurements such as Delta-e values and Nyquist Frequency, we expanded our technical vocabulary. We sought out resources to learn our new jargon like ImageZebra’s guide. We learned how to interpret the data and it became easier to iterate our physical setups to produce higher measuring results until we were able to meet our goals efficiently.

Software results in a chart showing colorimetric (LAB colorspace) results from a target tested for FADGI compliance. Details aren't checked and shown so it's an overview depicting all green 100% 4 star results in 9 specific categories starting at Lightness and ending at Noise.

What do we do when our testing results don't look, excitingly, like these 100% 4-star measuring shown above?

Here are some of the ways we've learned to interpret the data and respond to it:

Lightness / Tone Response

  • This is addressed via exposure -- our existing practices aligned closely with the target measurements for this metric. In order to decrease the distance in the measured and captured tonescale values in our photography setups, we have introduced s-curve adjustments in the luma channel and custom ICC profiles (custom color profile) when needed. We nearly always proceed with the latter solution since it has yielded more accurate results on our systems. Maintaining a consistent capture environment by eliminating reflections and light pollution is important for this metric as well. We have gray walls, blocked ceiling lights that are turned off, blackout shades, and gray or black lab coats when needed to help maintain a consistent and not-full-of-odd-reflections environment. We rarely use flatbed scanners in our workflow, but we are just now embarking on incorporating FADGI into baseline measurements for them in alignment with our other stations. Below is an example of how the visualized data from the target file images can help explain an image quality issue that is easy to see once you have something consistent to compare on the equipment (in this case, we tested a collaborating unit's flatbed scanner they use for reproductions for reference purposes to help resolve the issue of scans that appeared overly dark).

Delta E(a*b*) / White Balance

  • We white balance on the 13 patch of our item level targets, as much of our materials are lighter, this lighter gray value works well for us. We use linear scientific curves almost exclusively, resulting in a flat aesthetic with a lot of information. We rarely use linear responsive curves and when we do, it's for items that are highly reflective or there is clipping data (blown out) in the highlights. If any of this is off, this metric might show issues.
A gray rectangle with gray slant lines and optical resolution converging lines in the center and corners, black circles around edges and evenly spaced throughout, and many circles of colors, blues and greens near the center, skintones near the top, vibrant shades of orange and purple of varying shades near the bottom half. Lab color readouts are visible on the white patches, reading 248 on the square and 255 in all channels on the left of the gradient bar in the upper third of the rectangle and averaging 196 on patch E8.
A gray rectangle with gray slant lines and optical resolution converging lines in the center and corners, black circles around edges and evenly spaced throughout, and many circles of colors, blues and greens near the center, skintones near the top, vibrant shades of orange and purple of varying shades near the bottom half. Lab color readouts are visible on the white patches, reading 97.52 on the square and 99.805 in lightness on the left of the gradient bar in the upper third of the rectangle and 79.56 on patch E8.
Screenshot of exposures for our FADGI - ISO 19264 target with color readout values in LAB and sRGB colorspaces for a file used to create a custom ICC profile. We've found this file needs to be slightly brighter, by up to 8 points in sRGB, than our aim for validation targets or daily calibration targets. We use this dedicated FADGI device level target to create custom ICC profiles and validate, or check our performance with a different object level target.

Avg & 90% Color Accuracy - DeltaE 2000

  • This measures the difference between the measured colors on the targets we utilize and the measured L*ab values on the digitized image. Ideally, values reported in our image evaluation software should result in a low DeltaE 2000 value of less than 3 -- if when evaluated this value is above our accepted tolerance, we create a custom ICC profile for that scene or apply a more appropriate ICC profile. Additionally, we ensure that our white balance and exposures are accurate throughout the image.  As long as the tonescale metrics measure favorably, our cameras and lights can easily achieve this DeltaE 2000 metric consistently throughout capture sessions. Ensuring accurate focus, dusting/cleaning, and photographing at 1/125s shutter speed also helps minimize any ambient light interference with color reproduction.
A FADGI device level target which is a gray rectangle around the size of printer paper that displays gray slant edges in the center and corners, many colored circles and black dots throughout. This screenshot has numbers superimposed on the colorful circles representing Delta E values measured, most are under 5 but several teal and teal adjacent colors are over 10.

Delta E Value Measurements from a Flatbed Scanner illustrated here show some blue/teal areas suffered, as they measure higher than 10. Overall this target evaluates mostly Fadgi 3 star for color accuracy, which is good for most uses.

Color - Registration

  • This is rarely an issue and often resolves once other metrics are improved. The varying color channels will show different results as the software reads the horizontal and vertical regions of interest (ROIs). These also measure SFR (more on that later), so if these ROIs aren't aligned well in the initial test, that can also throw the measurements off. We do use Lens Cast Calibrations (LCCs) which are like flat fielding masks in other workflows – these tweak the files to even the lighting through processing steps in the software. LCCs (white or gray boards) need to be shot at the same aperture as the setup, and without any clipped (overly bright) areas. Careful LCC creation is important for these to function accurately and not negatively affect color (we adjust lights or shutter speed to darken the image around 1 stop to achieve intended exposure for these).

Sampling Frequency - Resolution

  • This specifically refers to the pixels per inch (ppi) the image was captured at (concerning the distance of the camera to the item in relation to the lens focal length and camera's sensor size), not what the file derivate was output as. For example, a 1 for 1 print of a 600ppi reproduction could be printed at 600dpi (dots per inch) and be accurately sized to the original. That same item could be digitized at a different resolution, either farther away or closer to the item, and output as a 600ppi file, but when that file is printed at 600dpi, it will be bigger or smaller than the original, depending on if the camera was closer or farther from the item, and will not be a 1 for 1 scale reproduction without resampling the file. You can resize a file's ppi to meet any number of uses, but the specific resolution in this metric is measured optically by the software looking at the dark gray dots on the targets being measured and calculating what ppi, in terms of distance to the item, the target was photographed at. 

Reproduction Scale Accuracy

  • This evaluation metric ties in closely with sampling frequency by comparing the output resolution (ppi) to the captured resolution (metric above). For example, a photograph would need to be captured at a minimum of 600ppi to be considered a FADGI 4-star image for this evaluation parameter. The amount in which the captured (spatial) ppi deviates from the exported ppi has an accepted tolerance of  + or - 6 pixels per inch for a FADGI 4-star image. The closer the captured ppi is to the target ppi, the higher the potential star rating would be for this metric. One can adjust any image file's ppi at any point in the workflow, but after the image is captured, it merely changes the density of pixels and does not increase the resolved detail in the image. 

10% SFR - Sampling Efficiency

  • This looks at the gray slant line areas of the tested targets. This Spatial Frequency Resolution has been improved for us by refining focus on the surface of the object, leveling the plane of the target to the plane of the camera sensor (we use a laser to do this), and shooting within the sharpest aperture for our camera systems. We've found f/6.3 creates the sharpest image, as compared to the smaller f/9 aperture which is prone to softness due to image diffraction. When using autofocus, the location of the target in the capture frame can also affect this metric -- focusing near a corner of a frame helps ensure tack sharp focus throughout our image edge to edge.
Two photographs zoomed into the center of an item level target where the optical resolution converging lines arranged vertically and horizontally that show dots per inch resolving power of an image illustrate that lines are distinctly visible past 600 on the left and only just past 500 on the right -- the gray slant lines to the bottom right on both have a left/top that's a darker gray portion of a vertical rectangle, and a lighter gray portion for the right/bottom portion.

Screenshot Detail Comparing Optical Image Resolution: the left side converging lines are visible farther into the 600ppi regions than the right side.

50% SFR (Spatial Frequency Response)

  • Is what we’re capturing optically as high a resolution as we think it is (in the graph below, reaching near the Nyquist Frequency)? This metric looks at the gray slant line areas of the tested targets. It has been a bit more challenging to rectify when measurements indicated something is off with this metric since it can be affected by a variety of in-camera and environmental factors. If this metric is performing poorly when analyzed, we first check the camera planarity with our laser level to ensure the camera and capture plane are perfectly aligned. Applying minimal sharpening in the post-processing phase is a tool we've tried and ultimately have not adopted to fix this metric. We add custom minimal sharpening to our access derivative files and remove all sharpening from our preservation file derivative TIFs. Artificial sharpening improves this metric, but we found through practice when all other metrics measure well this SFR does too, rendering the sharpening unnecessary. Added contrast helps: lens flare was shared with us as a culprit to poor SFR results from collaboration with other institutions, so we’ve also mitigated this with block card flags outside our lenses on our dual capture system. They function like extra big lens hoods, giving us a little boost in image quality. Limiting vibrations is helpful too, which could look as simple as slowing the pace of digitization down slightly, so movement doesn't negatively affect the images.
Two graphs, one titled Horizontal and the other Vertical, showing colorful lines representing aims for files measuring 2 star (orange), 3 star (yellow) and 4 star (green) in FADGI standards. The line is smooth, mostly linear from upper lefthand corner (Amplitude of 1 on the y-axis) to the Nyquist Freq dashed light to the bottom right of the graph at the 600dpi mark of the x-axis.

Spatial Frequency Response (SFR) Curve Graphs, the results here of a happy target image illustrated by the smooth data line reaching FADGI 4-star image goal posts.

Noise

  • When shooting at a higher ISO setting, such as we do at our dual capture system, our noise settings in software are minimally applied to all files. For other stations we can turn the noise adjustments off, as this has not been a metric we have had issues with. Given the high image quality our specialized camera equipment produces, noise is generally not a concern.

Testing Schedule

We now aim to measure at 80% or higher 4-Star Compliant at our stations. We test at the beginning of each project block (which can vary from weekly to each semester depending on the schedule and priorities). Our goal in testing is to ensure consistency, provide high quality imaging that's measurable, and monitor our equipment for potential maintenance needs.

 

The testing workflow looks something like this:

  1. Shoot object level target & output at preservation file specifications
  2. Move object level target file to server
  3. Download object level target from server onto remote Windows virtual machine
  4. Run object level target through software 
  5. Interpret analyzed results of object level target and adjust workspace as needed
  6. Retest as needed

 

Possible additional ICC profiling steps if that adjustment is needed:

  1. Shoot FADGI device level target
  2. Move object level target file to server
  3. Download object level target from server onto remote Windows virtual machine
  4. Run object level target through software and generate ICC profile 
  5. Move ICC profile into ColorSynch folder on workstation
  6. Restart Imaging software and apply new ICC profile to repeat testing steps above with an object level target

 

We shoot daily targets for every session and item level targets for each item we digitize. We have an internal practice of comparing our daily target's visually for focus, color and exposure to a "Best Bet Target" (one that tested well at the beginning of our setups that measured 80% compliant or better in our FADGI testing). Through this practice, we can have daily assurances we are accurate in our settings and setups, without slowing down daily to test and iterate, and each project we can work to improve our FADGI ratings and image quality. This close attention to detail has led us to finding a sharper aperture to shoot at, leading to better focus and more consistent evaluations of our equipment and performance.