David Baron's Weblog

Why adding compositing and blending to CSS is harder than it looks

Wednesday, 2013-03-06, 15:17 -0800

The Compositing and Blending specification (25 February 2013 editor's draft, latest editor's draft, latest TR draft) proposes the addition of compositing and blending operations to CSS. This proposal adds considerable graphical power to CSS, and could be quite useful. But doing this correctly for a Web with multiple implementations requires some caution. Explaining why this is the case requires a bit of explanation.

Drawing operations in CSS today use source-over compositing with normal blending. This combination has the property that drawing is associative (but not commutative). In other words, when you have three layers, say, rgba(255, 0, 0, 0.6) on top of rgba(128, 128, 0, 0.4) on top of blue, you get the same result (maybe modulo tiny variation for rounding) no matter how the drawing operators are grouped:

(rgba(255, 0, 0, 0.6) over rgba(128, 128, 0, 0.4)) over rgba(0, 0, 255, 1.0)
simplifies to
rgba(227, 26, 0, 0.76) over rgba(0, 0, 255, 1.0)
which simplifies to
rgba(173, 20, 61, 1.0).
rgba(255, 0, 0, 0.6) over (rgba(128, 128, 0, 0.4) over rgba(0, 0, 255, 1.0))
simplifies to
rgba(255, 0, 0, 0.6) over rgba(51, 51, 153, 1.0)
which simplifies to
rgba(173, 20, 61, 1.0).

But this no longer works if we change to a non-default compositing or blending mode. For example, let's replace one of those source-over operations with a source-atop compositing operation, and see what happens with different ways of grouping the operations.

(rgba(255, 0, 0, 0.6) atop rgba(128, 128, 0, 0.4)) over rgba(0, 0, 255, 1.0)
simplifies to
rgba(205, 50, 0, 0.4) over rgba(0, 0, 255, 1.0)
which simplifies to
rgba(82, 20, 153, 1.0).
rgba(255, 0, 0, 0.6) atop (rgba(128, 128, 0, 0.4) over rgba(0, 0, 255, 1.0))
simplifies to
rgba(255, 0, 0, 0.6) atop rgba(51, 51, 153, 1.0)
which simplifies to
rgba(173, 20, 61, 1.0).

Now, CSS specifies the order of drawing operations very carefully. But this specification of order was never a specification of grouping, and implementations have found different ways to group the drawing operations. Many optimizations that authors depend on (for example, that currently-animating elements with a transform set have their own layer that's stored and recomposited on the GPU without any repainting of that layer or the one it's composited into) are optimizations that are based on changing the grouping of drawing operations. Different browsers group drawing operations in different ways. For the most part, drawing is grouped from back to front, but other grouping is forced by opacity (where the specification requires grouping), and in some other cases for performance optimizations (which differ across browsers). [ Wording of last sentence updated 15:30. ]

Furthermore, the way the order of drawing operations in CSS was specified was never intended to be a specification of grouping. The specification was written in certain ways for the convenience of the specification authors or readers, but that manner of specification was not designed to produce good or intuitive results in cases where the grouping of drawing operations is exposed.

In order to specify blending and compositing throughout CSS in a way that will be interoperable across browsers, the grouping in CSS must be specified as clearly and with as much precision as the ordering is. This means that everything in Appendix E of CSS 2.1, plus all the additions to it from newer CSS modules that have not been fully tracked, needs to specify not only the ordering of the drawing operations but also their grouping.

If non-default compositing and blending is limited strictly to elements that create stacking contexts, as I have proposed (which means removing background-composite and background-blend-mode), then the specification problem becomes substantially easier, in that we at least only need to specify and interoperably implement the grouping of those of the drawing operations that involve elements creating stacking contexts, which means, I think, that the grouping would only need to be specified between:

  1. Together, items (1) through (2) in the top level of Appendix E
  2. Each item in item (3) in the top level of Appendix E
  3. Together, items (4) through (7) in the top level of Appendix E
  4. Each item in item (8) in the top level of Appendix E
  5. Each item in item (9) in the top level of Appendix E
  6. Together, (10) in the top level of Appendix E

(Some might object to this on the basis of the priority of constituencies. I believe such an objection is trivial to rebut: we don't have infinite resources and can't develop all features instantaneously. So giving authors a good solution now is often more valuable than a perfect solution later—potentially much later, if getting this specification right drops down in priority.)

There's still another issue, though, which is that the results the spec specifies need to be implementable efficiently (in terms of both performance and memory use) across multiple implementations. That requires not just specifying the rules clearly, but discussing them with implementors who understand the full rendering pipeline of different Web browser engines.

If these issues with the compositing and blending specification are ignored, then we risk either ending up with implementations in different browsers that just match the internals of those browsers, or unnecessarily delaying implementation of these features (or their usability on the Web, across multiple browsers).