An implementation of High Pass Skin Smoothing using Apple's Core Image Framework

Overview

YUCIHighPassSkinSmoothing

CocoaPods Platform CocoaPods Version CocoaPods License

An implementation of High Pass Skin Smoothing using CoreImage.framework

Available on both OS X and iOS.

Ports

A MetalPetal based version can be found here: MetalPetal/MTIHighPassSkinSmoothingFilter

A GPUImage based version can be found here: YUGPUImageHighPassSkinSmoothing

Previews

Preview 1

Preview 2

Preview 3

Preview 4

Preview 5

Preview 6

Performance

Tests are made in Instruments with the "OpenGL ES Analysis" template.

The CIContext object is created with EAGLContext and a sRGB working color space (CGColorSpaceCreateDeviceRGB()).

Image Size: 640 x 800
Input Radius: 7.0
Operating System: iOS 9

Device: iPhone 5s / FPS: 60
Device: iPhone 5  / FPS: ~24

Concepts

The basic routine of YUCIHighPassSkinSmoothing can be described with the following diagram.

Routine

Basic Concept

The main theory behind High Pass Skin Smoothing is Frequency Separation.

Frequency separation splits up the tones and colors of a image from its more detailed textures. It is possible because a digital image can be interpreted as different frequencies represented as sine waves.

High frequencies in an image will contain information about fine details, such as skin pores, hair, fine lines, skin imperfections.

Low frequencies are the image data that contains information about volume, tone and color transitions. In other words: shadows and light areas, colors and tones.

https://fstoppers.com/post-production/ultimate-guide-frequency-separation-technique-8699

By using High Pass filter, the image can be separated into high and low spatial frequencies. Then we will be able to smoothing the image while preseving a fine level of detail by applying adjustments (Curve Adjustment in the diagram) to certain frequencies of the image.

High Pass Filter

There's no High Pass filter in CoreImage. Luckily it's not hard to create one (High Pass Filter section in the diagram):

highpass.rgb = image.rgb - gaussianBlurredImage.rgb + vec3(0.5,0.5,0.5)

Mask Generating

A mask image is generated using high pass filter (Green and blue channel overlay blend -> high pass -> hard light blend X 3) for separating the parts of the image which need to be adjusted from the ones that should remain untouched.

This mask image is then used in Blend with Mask (CIBlendWithMask) filter to blend the adjusted image with the untouched one to produce the final output image.

Improvements

Besides the steps in the diagram, YUCIHighPassSkinSmoothing actually has two more steps.

The exposure of the input image is decreased by 1 EV before being sent to the Mask Generating Routine (in -[YUCIHighPassSkinSmoothingMaskGenerator outputImage] method) and a RGB curve adjustment is added to the mask at the end of the Mask Generating Routine (at the end of YUCIHighPassSkinSmoothingMaskBoost.cikernel).

These steps can make the result better on the areas with high brightness. The whole process can of course work without these two steps.

Input Parameters

inputAmount: A number value that controls the intensity of the Curve Adjustment step and affects the sharpness of the final Sharpen step. You use this value to control the overall filter strength. Valid from 0 to 1.0. The default value is 0.75.

inputToneCurveControlPoints: A array of CIVector that defines the control points of the curve in Curve Adjustment step. The default value of this parameter is [(0,0), (120/255.0,146/255.0), (1,1)].

inputRadius: A number value that controls the radius (in pixel) of the High Pass filter. The default value of this parameter is 8.0. Try adjusting this value according to the resolution of the input image and the level of detail you want to preserve.

inputSharpnessFactor: A number value that controls the sharpness factor of the final Sharpen step. The sharpness value is calculated as inputAmount * inputSharpnessFactor. The default value for this parameter is 0.6. Note: Sharpen is an expensive operation (it involves convolution). The final Sharpen step will make the output image look better, however it is not necessary. Consider setting this parameter to 0 when you need to get the best performance.

Consideration

For the best result, you need to create the CIContext object with a sRGB working color space instead of the default light-linear color space. You can specify a working color space when creating a context using the kCIContextWorkingColorSpace key in the options dictionary.

You can also try to apply this filter only to the skin/face area of a image, by using a skin detection filter or the core image face detector.

Requirements

  • iOS 8.0+ / OS X 10.11+
  • Xcode 7.0+

Usage

Use the YUCIHighPassSkinSmoothing, like any other built in core image filters.

Documentation for the input parameters can be found here.

let filter = YUCIHighPassSkinSmoothing()
filter.inputImage = ...
filter.inputAmount = ...
let outputImage = filter.outputImage!

/* Or */

let filter = CIFilter(name: "YUCIHighPassSkinSmoothing")!
filter.setValue(inputImage, forKey: kCIInputImageKey)
filter.setValue(0.7, forKey: "inputAmount")
let outputImage = filter.outputImage!

Open YUCIHighPassSkinSmoothingDemo/YUCIHighPassSkinSmoothingDemo.xcworkspace to run the iOS demo app. The demo app demonstrated how to use the filter and how to use different kinds of core image context to render the output image. The Metal core image context is only available on 64-bit devices with iOS 9.

Please run the demo app on an actual device. The iOS simulator does not provide a pixel-accurate match to the graphics hardware. Rendering performance of OpenGL ES in Simulator has no relation to the performance of OpenGL ES on an actual device.

Installation

YUCIHighPassSkinSmoothing makes use of Vivid.

CocoaPods

Add the following to your Podfile

pod 'YUCIHighPassSkinSmoothing'

Manually

Clone this repo and Vivid, then manually add the files in Sources directories to your project.

Contributing

You are encouraged to try different input parameters or tweak the interal procedure to make this filter better or just fit your needs.

Don't hesitate to open an issue if you have any idea or suggestion.

If you find a bug and know exactly how to fix it, please open a pull request. Be sure to test the code on both OS X and iOS.

Credits

Thanks a lot to Yien Ma for providing a lot of suggestions and fine-tunings to the procedure.

License

YUCIHighPassSkinSmoothing is MIT-licensed. See LICENSE file for detail.

Copyright © 2016 Yu Ao

Issues
  • use too much cpu

    use too much cpu

    opened by notedit 13
  • Love this project, but is it possible to implement this filter by GPUImage?

    Love this project, but is it possible to implement this filter by GPUImage?

    When I use this filter to process live video buffer, cpu usage can goes up to 70% . Is it possible to reduce the cpu usage to under 15%, 25 frames per second. Besides, is GPUImage able to imitate the same effect with this great filter?

    opened by jinbo0122 3
  • let filter = CIFilter(name:

    let filter = CIFilter(name: "YUCIHighPassSkinSmoothing")! Crash

    I am trying to access YUCIHighPassSkinSmoothing filter with CIFilter but I am not able to access it.

    I am using the same code which you have provided in the sample like below:

    let filter = CIFilter(name: "YUCIHighPassSkinSmoothing")!

    Thanks!

    opened by yashesh87 3
  • Hi

    Hi

    Very good job, thank you very much!

    Could you tell me where to find a skin detection filter ?

    opened by Tippit 2
  • principle of skinSmooth

    principle of skinSmooth

    opened by xinlinxie 2
  • Cannot use sRGB texture binding with format BGRA8 for sampler 0 in kernel 'filterKernel'.

    Cannot use sRGB texture binding with format BGRA8 for sampler 0 in kernel 'filterKernel'.

    Hi YuAo , your work is fantastic . ( Unfortunately im not smart enough to use it ). Failed to use RGBToneCurve to filtering live iOS camera. I would appreciate for your help . Thanks.

    https://stackoverflow.com/questions/49933477/ios-yucirgbtonecurve-cannot-use-srgb-texture-binding-with-format-bgra8-for-sam

    opened by gazind 2
  • How to implement this beautify filter in live camera to beautify face

    How to implement this beautify filter in live camera to beautify face

    Hi @yien1108 , @YuAo ,

    I tried GPUImage2, but there is no method to beautify face in the live camera. Can you please help how to implement this filter with the camera?

    Thanks in advance.

    opened by aaasifrz9 2
  • Apply filter on selected part of image

    Apply filter on selected part of image

    How to apply that YUCIHighPassSkinSmoothing filter on selected/specific part of image, like pass a particular portion of image from image to apply filter on it, i have tried this let imageCenter = CIVector(x:0.55 * 100 , y:0.6 * 100) self.filter.setValue(imageCenter, forKey:"inputCenter") But it says "this class is not key value coding-compliant for the key inputCenter" I have applied above inputCenter in processImage() method of class DefaultRenderContextViewController.

    opened by kk143g 1
  • inputToneCurveControlPoints invalid name in README

    inputToneCurveControlPoints invalid name in README

    Hi, thanks for this great library!

    I've noticed that there's a parameter called inputControlPoints in the README, but it's actually called inputToneCurveControlPoints in the code, so trying to set inputControlPoints param leads to exception.

    Also where I can read what exactly is "curve from the Curve Adjustment step" and what its values are responsible for? Thanks!

    opened by RomanFrom710 1
  • How to use this Feature with Live Camera

    How to use this Feature with Live Camera

    I want to use this feature with live camera, How can I Implement this? Can you please help?

    opened by khush004 1
  • How to get smoothening effect like YUGPUImageStillImageHighPassFilter does? YUCIHighPassSkinSmoothing makes the image too much bright.

    How to get smoothening effect like YUGPUImageStillImageHighPassFilter does? YUCIHighPassSkinSmoothing makes the image too much bright.

    YUCIHighPassSkinSmoothing makes the image too much bright. YUGPUImageStillImageHighPassFilter creates the blush effect.

    opened by shiwanithakur 0
  • LowPass Skin Smoothing

    LowPass Skin Smoothing

    Hi, thanks for creating this awesome repository.

    I am a newbie at writing shader string, so can you please suggest me a way to target the low-frequency areas in the image just like you are creating the high pass filter in the following way, to target the high frequencies areas of the image

    highpass.rgb = image.rgb - gaussianBlurredImage.rgb + vec3(0.5,0.5,0.5)

    Any help would be highly appreciated. Thanks

    opened by swakez 0
Releases(1.4)
Owner
Yu Ao
Yu Ao
📷 A composable image editor using Core Image and Metal.

Brightroom - Composable image editor - building your own UI Classic Image Editor PhotosCrop Face detection Masking component ?? v2.0.0-alpha now open!

Muukii 2.5k Jan 14, 2022
High Quality Image ScrollView using cropped tiled images.

THTiledImageView Feature ?? THTiledImageView fully support UIScrollView. You can subclass it and use it. ?? Support Async Image Downloading & Caching.

null 28 Apr 4, 2021
An extremely high-performance, lightweight, and energy-efficient pure Swift async web image loader with memory and disk caching for iOS and  Watch.

KFSwiftImageLoader KFSwiftImageLoader is an extremely high-performance, lightweight, and energy-efficient pure Swift async web image loader with memor

Kiavash Faisali 343 Sep 20, 2021
A high-performance image library for downloading, caching, and processing images in Swift.

Features Asynchronous image downloader with priority queuing Advanced memory and database caching using YapDatabase (SQLite) Guarantee of only one ima

Yap Studios 71 Nov 14, 2021
Twitter Image Pipeline is a robust and performant image loading and caching framework for iOS clients

Twitter Image Pipeline (a.k.a. TIP) Background The Twitter Image Pipeline is a streamlined framework for fetching and storing images in an application

Twitter 1.8k Dec 27, 2021
Syntactic Sugar for Accelerate/vImage and Core Image Filters

ShinpuruImage Syntactic Sugar for Accelerate/vImage and Core Image Filters ShinpuruImage offers developers a consistent and strongly typed interface t

simon gladman 98 Dec 11, 2021
GPUImage 3 is a BSD-licensed Swift framework for GPU-accelerated video and image processing using Metal.

GPUImage 3 Janie Clayton http://redqueengraphics.com @RedQueenCoder Brad Larson http://www.sunsetlakesoftware.com @bradlarson [email protected]

Brad Larson 2.3k Jan 4, 2022
An image download extension of the image view written in Swift for iOS, tvOS and macOS.

Moa, an image downloader written in Swift for iOS, tvOS and macOS Moa is an image download library written in Swift. It allows to download and show an

Evgenii Neumerzhitckii 327 Oct 29, 2021
High performance and delightful way to play with APNG format in iOS.

APNGKit is a high performance framework for loading and displaying APNG images in iOS and macOS. It's built on top of a modified version of libpng wit

Wei Wang 1.9k Jan 14, 2022
High-performance animated GIF support for iOS in Swift

Gifu adds protocol-based, performance-aware animated GIF support to UIKit. (It's also a prefecture in Japan). Install Swift Package Manager Add the fo

Reda Lemeden 2.7k Jan 11, 2022
High performance GIF engine

SwiftyGif High performance & easy to use Gif engine Features UIImage and UIImageView extension based Remote GIFs with customizable loader Great CPU/Me

Alexis Creuzot 1.5k Jan 5, 2022
A simple UIImageView extension for using initials as a profile image, written in swift

InitialsImageView An easy, helpful UIImageView extension that generates letter initials as a placeholder for user profile images, with a randomized ba

Tom Bachant 211 Dec 29, 2021
iOS hashtag generator, using image analysis and discovery

Tagger Description Want to be popular on some social network easily? Use Tagger to make your account content more popular and to raise your popularity

Ivan Magda 40 Mar 11, 2021
An open source iOS framework for GPU-based image and video processing

GPUImage Brad Larson http://www.sunsetlakesoftware.com @bradlarson [email protected] Overview The GPUImage framework is a BSD-licensed iO

Brad Larson 19.9k Jan 10, 2022
Asynchronous image loading framework.

YYWebImage YYWebImage is an asynchronous image loading framework (a component of YYKit). It was created as an improved replacement for SDWebImage, PIN

null 3.5k Jan 5, 2022
GPUImage 2 is a BSD-licensed Swift framework for GPU-accelerated video and image processing.

GPUImage 2 Brad Larson http://www.sunsetlakesoftware.com @bradlarson [email protected] Overview GPUImage 2 is the second generation of th

Brad Larson 4.7k Jan 4, 2022
An image compositing framework written in Swift.

An asynchronous, multithreaded, image compositing framework written in Swift. Installation CocoaPods Add Overlay to your Podfile: pod 'OverlayComposit

Aaron Sutton 12 Dec 18, 2021
A GPU accelerated image and video processing framework built on Metal.

MetalPetal An image processing framework based on Metal. Design Overview Goals Core Components MTIContext MTIImage MTIFilter MTIKernel Alpha Type Hand

null 1.2k Jan 9, 2022
SwiftUI Image loading and Animation framework powered by SDWebImage

SDWebImageSwiftUI What's for SDWebImageSwiftUI is a SwiftUI image loading framework, which based on SDWebImage. It brings all your favorite features f

null 1.2k Jan 12, 2022