# Writing and using fragment shaders > How to author and use fragment shaders to create custom visual effects in your Flutter app. :::note Both the Skia and [Impeller][] backends support writing a custom shader. Except where noted, the same instructions apply to both. ::: [Impeller]: /perf/impeller Custom shaders can be used to provide rich graphical effects beyond those provided by the Flutter SDK. A shader is a program authored in a small, Dart-like language, known as GLSL, and executed on the user's GPU. Custom shaders are added to a Flutter project by listing them in the `pubspec.yaml` file, and obtained using the [`FragmentProgram`][] API. [`FragmentProgram`]: https://api.flutter.dev/flutter/dart-ui/FragmentProgram-class.html ## Adding shaders to an application Shaders, in the form of GLSL files with the `.frag` extension, must be declared in the `shaders` section of your project's `pubspec.yaml` file. The Flutter command-line tool compiles the shader to its appropriate backend format, and generates its necessary runtime metadata. The compiled shader is then included in the application just like an asset. ```yaml flutter: shaders: - shaders/myshader.frag ``` When running in debug mode, changes to a shader program trigger recompilation and update the shader during hot reload or hot restart. Shaders from packages are added to a project with `packages/$pkgname` prefixed to the shader program's name (where `$pkgname` is the name of the package). ### Loading shaders at runtime To load a shader into a `FragmentProgram` object at runtime, use the [`FragmentProgram.fromAsset`][] constructor. The asset's name is the same as the path to the shader given in the `pubspec.yaml` file. [`FragmentProgram.fromAsset`]: https://api.flutter.dev/flutter/dart-ui/FragmentProgram/fromAsset.html ```dart void loadMyShader() async { var program = await FragmentProgram.fromAsset('shaders/myshader.frag'); } ``` The `FragmentProgram` object can be used to create one or more [`FragmentShader`][] instances. A `FragmentShader` object represents a fragment program along with a particular set of _uniforms_ (configuration parameters). The available uniforms depends on how the shader was defined. [`FragmentShader`]: https://api.flutter.dev/flutter/dart-ui/FragmentShader-class.html ```dart void updateShader(Canvas canvas, Rect rect, FragmentProgram program) { var shader = program.fragmentShader(); shader.setFloat(0, 42.0); canvas.drawRect(rect, Paint()..shader = shader); } ``` ### Canvas API Fragment shaders can be used with most Canvas APIs by setting [`Paint.shader`][]. For example, when using [`Canvas.drawRect`][] the shader is evaluated for all fragments within the rectangle. For an API like [`Canvas.drawPath`][] with a stroked path, the shader is evaluated for all fragments within the stroked line. Some APIs, such as [`Canvas.drawImage`][], ignore the value of the shader. [`Canvas.drawImage`]: https://api.flutter.dev/flutter/dart-ui/Canvas/drawImage.html [`Canvas.drawRect`]: https://api.flutter.dev/flutter/dart-ui/Canvas/drawRect.html [`Canvas.drawPath`]: https://api.flutter.dev/flutter/dart-ui/Canvas/drawPath.html [`Paint.shader`]: https://api.flutter.dev/flutter/dart-ui/Paint/shader.html ```dart void paint(Canvas canvas, Size size, FragmentShader shader) { // Draws a rectangle with the shader used as a color source. canvas.drawRect( Rect.fromLTWH(0, 0, size.width, size.height), Paint()..shader = shader, ); // Draws a stroked rectangle with the shader only applied to the fragments // that lie within the stroke. canvas.drawRect( Rect.fromLTWH(0, 0, size.width, size.height), Paint() ..style = PaintingStyle.stroke ..shader = shader, ) } ``` ### ImageFilter API Fragment shaders can also be used with the [`ImageFilter`][] API. This allows using custom fragment shaders with the [`ImageFiltered`][] class or the [`BackdropFilter`][] class to apply shaders to already rendered content. [`ImageFilter`][] provides a constructor, [`ImageFilter.shader`][], for creating an [`ImageFilter`][] with a custom fragment shader. :::warning The `ImageFilter` API for custom shaders is only supported by the [Impeller][] backend. Using it on other backends will throw an error. ::: ```dart Widget build(BuildContext context, FragmentShader shader) { return ClipRect( child: SizedBox( width: 300, height: 300, child: BackdropFilter( filter: ImageFilter.shader(shader), child: Container( color: Colors.transparent, ), ), ), ); } ``` When using [`ImageFilter`][] with [`BackdropFilter`][], a [`ClipRect`][] can be used to limit the area that is affected by the [`ImageFilter`][]. Without a [`ClipRect`][] the [`BackdropFilter`][] will be applied to the whole screen. `ImageFilter` fragment shaders receive some uniforms automatically from the engine. The `sampler2D` value at index 0 is set to the filter input image, and the `float` values at indices 0 and 1 are set to the image's width and height. Your shader must specify this constructor to accept these values (for example, a `sampler2D` and a `vec2`), but you should not set them from your Dart code. When targeting OpenGLES the y-coordinates of the texture will be flipped so the fragment shader should un-flip the UVs when sampling from textures provided by the engine. ```glsl #version 460 core #include out vec4 fragColor; // These uniforms are automatically set by the engine. uniform vec2 u_size; uniform sampler2D u_texture; void main() { vec2 uv = FlutterFragCoord().xy / u_size; #ifdef IMPELLER_TARGET_OPENGLES // When sampling from u_texture on OpenGLES the y-coordinates will be flipped. uv.y = 1.0 - uv.y; #endif vec4 color = texture(u_texture, uv); float gray = dot(color.rgb, vec3(0.299, 0.587, 0.114)); fragColor = vec4(vec3(gray), color.a); } ``` [`ImageFilter`]: https://api.flutter.dev/flutter/dart-ui/ImageFilter-class.html [`ImageFiltered`]: https://api.flutter.dev/flutter/widgets/ImageFiltered-class.html [`BackdropFilter`]: https://api.flutter.dev/flutter/widgets/BackdropFilter-class.html [`ImageFilter.shader`]: https://api.flutter.dev/flutter/dart-ui/ImageFilter/ImageFilter.shader.html [`ClipRect`]: https://api.flutter.dev/flutter/widgets/ClipRect-class.html ## Authoring shaders Fragment shaders are authored as GLSL source files. By convention, these files have the `.frag` extension. (Flutter doesn't support vertex shaders, which would have the `.vert` extension.) Any GLSL version from 460 down to 100 is supported, though some available features are restricted. The rest of the examples in this document use version `460 core`. Shaders are subject to the following limitations when used with Flutter: * UBOs and SSBOs aren't supported * `sampler2D` is the only supported sampler type * Only the two-argument version of `texture` (sampler and uv) is supported * No additional varying inputs can be declared * All precision hints are ignored when targeting Skia * Unsigned integers and booleans aren't supported ### Uniforms A fragment program can be configured by defining `uniform` values in the GLSL shader source and then setting these values in Dart for each fragment shader instance. Floating point uniforms with the GLSL types `float`, `vec2`, `vec3`, and `vec4` are set using the [`FragmentShader.setFloat`][] or [`FragmentShader.getUniformFloat`][] method. GLSL sampler values, which use the `sampler2D` type, are set using the [`FragmentShader.setImageSampler`][] or [`FragmentShader.getImageSampler`][] method. The correct index for each `uniform` value is determined by the order that the uniform values are defined in the fragment program. For data types composed of multiple floats, such as a `vec4`, you must call [`FragmentShader.setFloat`][] or [`UniformFloatSlot.set`][] once for each value. [`FragmentShader.setFloat`]: https://api.flutter.dev/flutter/dart-ui/FragmentShader/setFloat.html [`UniformFloatSlot.set`]: https://api.flutter.dev/flutter/dart-ui/UniformFloatSlot/set.html [`FragmentShader.getUniformFloat`]: https://api.flutter.dev/flutter/dart-ui/FragmentShader/getUniformFloat.html [`FragmentShader.setImageSampler`]: https://api.flutter.dev/flutter/dart-ui/FragmentShader/setImageSampler.html [`FragmentShader.getImageSampler`]: https://api.flutter.dev/flutter/dart-ui/FragmentShader/getImageSampler.html For example, given the following uniforms declarations in a GLSL fragment program: ```glsl uniform float uScale; uniform sampler2D uTexture; uniform vec2 uMagnitude; uniform vec4 uColor; ``` The corresponding Dart code to initialize these `uniform` values is as follows: ```dart class Foobar { late final UniformFloatSlot _scale; late final List _magnitude; late final List _color; late final ImageSamplerSlot _texture; void setUp(FragmentShader shader) { _scale = shader.getUniformFloat('uScale'); _magnitude = List.generate(2, (int index) { return shader.getUniformFloat('uMagnitude', index); }); _color = List.generate(4, (int index) { return shader.getUniformFloat('uColor', index); }); _texture = shader.getImageSampler('uTexture'); } void update(Color color, Image image) { _scale.set(23); _magnitude[0].set(114); _magnitude[1].set(83); _color[0].set(color.r * color.a); _color[1].set(color.g * color.a); _color[2].set(color.b * color.a); _color[3].set(color.a); _texture.set(image); } } ``` When using [`FragmentShader.setFloat`][] note that the indices do not count the `sampler2D` uniform. This uniform is set separately with [`FragmentShader.setImageSampler`][], with the index starting over at 0. Any float uniforms that are left uninitialized will default to `0.0`. The reflection data generated by the Flutter's shader compiler can be audited with the following commands in order to see things like uniform offsets. ```shell cd $FLUTTER # Generate the .sl file. `find bin/ -name impellerc` \ --runtime-stage-metal \ --iplr \ --input=path/to/myshader.frag \ --sl=foo.sl \ --spirv=foo.spirv \ --include=engine/src/flutter/impeller/compiler/shader_lib/ \ --input-type=frag # Convert the .sl file to .json flatc \ --json \ ./engine/src/flutter/impeller/runtime_stage/runtime_stage.fbs \ -- ./foo.sl # View results cat foo.json ``` #### Current position The shader has access to a `varying` value that contains the local coordinates for the particular fragment being evaluated. Use this feature to compute effects that depend on the current position, which can be accessed by importing the `flutter/runtime_effect.glsl` library and calling the `FlutterFragCoord` function. For example: ```glsl #include void main() { vec2 currentPos = FlutterFragCoord().xy; } ``` The value returned from `FlutterFragCoord` is distinct from `gl_FragCoord`. `gl_FragCoord` provides the screen space coordinates and should generally be avoided to ensure that shaders are consistent across backends. When targeting a Skia backend, the calls to `gl_FragCoord` are rewritten to access local coordinates but this rewriting isn't possible with Impeller. #### Colors There isn't a built-in data type for colors. Instead they are commonly represented as a `vec4` with each component corresponding to one of the RGBA color channels. The single output `fragColor` expects that the color value is normalized to be in the range of `0.0` to `1.0` and that it has premultiplied alpha. This is different than typical Flutter colors which use a `0-255` value encoding and have unpremultipled alpha. #### Samplers A sampler provides access to a `dart:ui` `Image` object. This image can be acquired either from a decoded image or from part of the application using [`Scene.toImageSync`][] or [`Picture.toImageSync`][]. [`Picture.toImageSync`]: https://api.flutter.dev/flutter/dart-ui/Picture/toImageSync.html [`Scene.toImageSync`]: https://api.flutter.dev/flutter/dart-ui/Scene/toImageSync.html ##### Sampler usage in GLSL example ```glsl #include uniform vec2 uSize; uniform sampler2D uTexture; out vec4 fragColor; void main() { vec2 uv = FlutterFragCoord().xy / uSize; fragColor = texture(uTexture, uv); } ``` By default, the image uses [`TileMode.clamp`][] to determine how values outside of the range of `[0, 1]` behave. Customization of the tile mode is not supported and needs to be emulated in the shader. [`TileMode.clamp`]: https://api.flutter.dev/flutter/dart-ui/TileMode.html ##### `toImageSync` example ```dart class SDFPainter { SDFPainter(this.sdfShader, this.renderShader); FragmentShader sdfShader; FragmentShader renderShader; Image? _sdf; bool isDirty = false; double radius = 0.5; void paint(Canvas canvas, Size size) { if (_sdf == null || isDirty) { final recorder = PictureRecorder(); final subCanvas = Canvas(recorder); final paint = Paint()..shader = sdfShader; sdfShader.setFloat(0, size.width); sdfShader.setFloat(1, size.height); sdfShader.setFloat(2, radius); subCanvas.drawRect(Rect.fromLTWH(0, 0, size.width, size.height), paint); final picture = recorder.endRecording(); _sdf = picture.toImageSync(size.width.toInt(), size.height.toInt()); isDirty = false; } renderShader.setFloat(0, size.width); renderShader.setFloat(1, size.height); renderShader.setImageSampler(0, _sdf!); canvas.drawRect( Rect.fromLTWH(0, 0, size.width, size.height), Paint()..shader = renderShader, ); } } ``` ## Performance considerations When targeting the Skia backend, loading the shader might be expensive since it must be compiled to the appropriate platform-specific shader at runtime. If you intend to use one or more shaders during an animation, consider precaching the fragment program objects before starting the animation. You can reuse a `FragmentShader` object across frames; this is more efficient than creating a new `FragmentShader` for each frame. For a more detailed guide on writing performant shaders, check out [Writing efficient shaders][] on GitHub. [Writing efficient shaders]: https://github.com/flutter/flutter/blob/main/docs/engine/impeller/docs/shader_optimization.md ## Other resources For more information, here are a few resources. * [The Book of Shaders][] by Patricio Gonzalez Vivo and Jen Lowe * [Shader toy][], a collaborative shader playground * [`simple_shader`][], a simple Flutter fragment shaders sample project * [`flutter_shaders`][], a package that simplifies using fragment shaders in Flutter [Shader toy]: https://www.shadertoy.com/ [The Book of Shaders]: https://thebookofshaders.com/ [`simple_shader`]: https://github.com/flutter/samples/tree/main/simple_shader [`flutter_shaders`]: https://pub.dev/packages/flutter_shaders