Proper transparent blitter stacking

This completes the work for #1068. This addressed a subtle issue.
When we're using pixel->semigraphic art, we want slightly different
rendering. Essentially, imagine that we have two images, each two
pixels tall and one pixel wide. The top image is a transparent pixel
above a white pixel. The bottom image is a white pixel above a black
pixel. We'd expect the result to be two white pixels, but we can
instead get a black pixel above a white pixel. This is because the
*background* color is being merged from the bottom plane, but really
we want the *top* color. Ncvisuals are now blitted along with
information regarding which quadrants they draw over, and when
appropriate, we invert the foreground and background. Closes #1068.

doc/man/man3/notcurses_render.3.md

@@ -121,5 +121,6 @@ purposes of color blending.
 **notcurses_plane(3)**,
 **notcurses_refresh(3)**,
 **notcurses_stats(3)**,
+**notcurses_visual(3)**,
 **console_codes(4)**,
 **utf-8(7)**

doc/man/man3/notcurses_visual.3.md

@@ -159,7 +159,39 @@ The different **ncblitter_e** values select from among available glyph sets:
 * **NCBLIT_SIXEL**: Not yet implemented.
 
 **NCBLIT_4x1** and **NCBLIT_8x1** are intended for use with plots, and are
-not really applicable for general visuals.
+not really applicable for general visuals. **NCBLIT_BRAILLE** doesn't tend
+to work out very well for images, but (depending on the font) can be very
+good for plots.
+
+In the absence of scaling, for a given set of pixels, more rows and columns in
+the blitter will result in a smaller output image. An image rendered with
+**NCBLIT_1x1** will be twice as tall as the same image rendered with
+**NCBLIT_2x1**, which will be twice as wide as the same image rendered with
+**NCBLIT_2x2**. The same image rendered with **NCBLIT_3x2** will be one-third
+as tall and one-half as wide as the original **NCBLIT_1x1** render (again, this
+depends on **NCSCALE_NONE**). If the output size is held constant (using for
+instance **NCSCALE_SCALE_HIRES** and a large image), more rows and columns will
+result in more effective resolution.
+
+Assuming a cell is twice as tall as it is wide, **NCBLIT_1x1** (and indeed
+any NxN blitter) will stretch an image by a factor of 2 in the vertical
+dimension. **NCBLIT_2x1** will not distort the image whatsoever, as it maps a
+vector two pixels high and one pixel wide to a single cell. **NCBLIT_3x2** will
+stretch an image by a factor of 1.5.
+
+The cell's dimension in pixels is ideally evenly divisible by the blitter
+geometry. If **NCBLIT_3x2** is used together with a cell 8 pixels wide and
+14 pixels tall, two of the vertical segments will be 5 pixels tall, while one
+will be 4 pixels tall. Such unequal distributions are more likely with larger
+blitter geometries. Likewise, there are only ever two colors available to us in
+a given cell. **NCBLIT_1x1** and **NCBLIT_2x2** can be perfectly represented
+with two colors per cell. Blitters of higher geometry are increasingly likely
+to require some degree of interpolation. Transparency is always honored with
+complete fidelity.
+
+Finally, rendering operates slightly differently when two planes have both been
+blitted, and one lies atop the other. See **notcurses_render(3)** for more
+information.
 
 # RETURN VALUES
 
@@ -210,11 +242,14 @@ radians for **rads**, but this will change soon.
 among terminals.
 
 Bad font support can ruin **NCBLIT_2x2**, **NCBLIT_3x2**, **NCBLIT_4x1**,
-**NCBLIT_BRAILLE**, and **NCBLIT_8x1**.
+**NCBLIT_BRAILLE**, and **NCBLIT_8x1**. Braille glyphs ought ideally draw only
+the raised dots, rather than drawing all eight dots with two different styles.
+It's often best for the emulator to draw these glyphs itself.
 
 # SEE ALSO
 
 **notcurses(3)**,
 **notcurses_capabilities(3)**,
 **notcurses_plane(3)**,
+**notcurses_render(3)**,
 **utf-8(7)**

src/lib/blit.c

@@ -391,6 +391,7 @@ qtrans_check(nccell* c, bool blendcolors,
   }else if(blendcolors){
     cell_set_fg_alpha(c, CELL_ALPHA_BLEND);
   }
+//fprintf(stderr, "QBQ: 0x%x\n", cell_blittedquadrants(c));
   return egc;
 }
 
@@ -610,6 +611,7 @@ sex_trans_check(cell* c, const uint32_t rgbas[6], bool blendcolors){
     cell_set_blitquadrants(c, !(transstring & 5u), !(transstring & 10u),
                               !(transstring & 20u), !(transstring & 40u));
   }
+//fprintf(stderr, "SEX-BQ: 0x%x\n", cell_blittedquadrants(c));
   return egc;
 }
 

src/lib/internal.h

@@ -852,11 +852,15 @@ cell_nobackground_p(const nccell* c){
   return (c->channels & CELL_NOBACKGROUND_MASK) == CELL_NOBACKGROUND_MASK;
 }
 
-// True iff the cell was blitted as part of an ncvisual, and has a transparent
-// background (being the only case where CELL_BLITTERSTACK_MASK bits are set).
-static inline bool
-cell_blitted_p(const nccell* c){
-  return c->channels & CELL_BLITTERSTACK_MASK; // any of the four bits is fine
+// Returns a number 0 <= n <= 15 representing the four quadrants, and which (if
+// any) are occupied due to blitting with a transparent background. The mapping
+// is {tl, tr, bl, br}.
+static inline unsigned
+cell_blittedquadrants(const nccell* c){
+  return ((c->channels & 0x8000000000000000ull) ? 1 : 0) |
+         ((c->channels & 0x0400000000000000ull) ? 2 : 0) |
+         ((c->channels & 0x0200000000000000ull) ? 4 : 0) |
+         ((c->channels & 0x0100000000000000ull) ? 8 : 0);
 }
 
 // Set this whenever blitting an ncvisual, when we have a transparent
@@ -866,10 +870,10 @@ static inline void
 cell_set_blitquadrants(nccell* c, unsigned tl, unsigned tr, unsigned bl, unsigned br){
   // FIXME want a static assert that these four constants OR together to
   // equal CELL_BLITTERSTACK_MASK, bah
-  c->channels |= tl ? 0x8000000000000000ull : 0;
-  c->channels |= tr ? 0x0400000000000000ull : 0;
-  c->channels |= bl ? 0x0200000000000000ull : 0;
-  c->channels |= br ? 0x0100000000000000ull : 0;
+  c->channels |= (tl ? 0x8000000000000000ull : 0);
+  c->channels |= (tr ? 0x0400000000000000ull : 0);
+  c->channels |= (bl ? 0x0200000000000000ull : 0);
+  c->channels |= (br ? 0x0100000000000000ull : 0);
 }
 
 // Destroy a plane and all its bound descendants.

src/lib/render.c

@@ -176,23 +176,26 @@ struct crender {
   // and then reapply any foreground shading from above the highcontrast
   // declaration. save the foreground state when we go highcontrast.
   unsigned hcfgblends; // number of foreground blends prior to HIGHCONTRAST
+  // FIXME can't hcfg be reduced to 24 bits and shoved in the bitfield below?
   uint32_t hcfg;       // foreground channel prior to HIGHCONTRAST
-  bool damaged;        // only used in rasterization
-  // if CELL_ALPHA_HIGHCONTRAST is in play, we apply the HSV flip once the
-  // background is locked in. set highcontrast to indicate this.
-  bool highcontrast;
-  // If the glyph we render is from an ncvisual, and has a transparent or
-  // blended background, blitter stacking is in effect. This is a complicated
-  // issue, but essentially, imagine a bottom block is rendered with a green
-  // bottom and transparent top. on a lower plane, a top block is rendered with
-  // a red foreground and blue background. Normally, this would result in a
-  // blue top and green bottom, but that's not what we ever wanted -- what makes
-  // sense is a red top and green bottom. So ncvisual rendering sets
-  // CELL_BLITTERSTACK_MASK when rendering a cell with a transparent background.
-  // When paint() selects a glyph, it checks for this flag. If the flag is set,
-  // any lower planes with CELL_BLITTERSTACK_MASK set take this into account
-  // when solving the background.
-  bool blitterstacked;
+  struct {
+    // If the glyph we render is from an ncvisual, and has a transparent or
+    // blended background, blitter stacking is in effect. This is a complicated
+    // issue, but essentially, imagine a bottom block is rendered with a green
+    // bottom and transparent top. on a lower plane, a top block is rendered
+    // with a red foreground and blue background. Normally, this would result
+    // in a blue top and green bottom, but that's not what we ever wanted --
+    // what makes sense is a red top and green bottom. So ncvisual rendering
+    // sets bits from CELL_BLITTERSTACK_MASK when rendering a cell with a
+    // transparent background. When paint() selects a glyph, it checks for these
+    // bits. If they are set, any lower planes with CELL_BLITTERSTACK_MASK set
+    // take this into account when solving the background color.
+    unsigned blittedquads: 4;
+    unsigned damaged: 1; // only used in rasterization
+    // if CELL_ALPHA_HIGHCONTRAST is in play, we apply the HSV flip once the
+    // background is locked in. set highcontrast to indicate this.
+    unsigned highcontrast: 1;
+  } s;
 };
 
 // Emit fchannel with RGB changed to contrast effectively against bchannel.
@@ -287,7 +290,7 @@ paint(const ncplane* p, struct crender* rvec, int dstleny, int dstlenx,
           }
         }else{
           if(cell_fg_alpha(vis) == CELL_ALPHA_HIGHCONTRAST){
-            crender->highcontrast = true;
+            crender->s.highcontrast = true;
             crender->hcfgblends = crender->fgblends;
             crender->hcfg = cell_fchannel(targc);
           }
@@ -295,7 +298,7 @@ paint(const ncplane* p, struct crender* rvec, int dstleny, int dstlenx,
           // crender->highcontrast can only be true if we just set it, since we're
           // about to set targc opaque based on crender->highcontrast (and this
           // entire stanza is conditional on targc not being CELL_ALPHA_OPAQUE).
-          if(crender->highcontrast){
+          if(crender->s.highcontrast){
             cell_set_fg_alpha(targc, CELL_ALPHA_OPAQUE);
           }
         }
@@ -308,8 +311,7 @@ paint(const ncplane* p, struct crender* rvec, int dstleny, int dstlenx,
       // Evaluate the background first, in case we have HIGHCONTRAST fg text.
       if(cell_bg_alpha(targc) > CELL_ALPHA_OPAQUE){
         const nccell* vis = &p->fb[nfbcellidx(p, y, x)];
-        // FIXME need check maps to determine whether inversion is appropriate
-        if(!crender->blitterstacked || !cell_blitted_p(vis)){
+        if(!((!crender->s.blittedquads) & cell_blittedquadrants(vis))){
           if(cell_bg_default_p(vis)){
             vis = &p->basecell;
           }
@@ -331,6 +333,7 @@ paint(const ncplane* p, struct crender* rvec, int dstleny, int dstlenx,
           }else{
             cell_blend_bchannel(targc, cell_fchannel(vis), &crender->bgblends);
           }
+          crender->s.blittedquads = 0;
         }
       }
 
@@ -349,7 +352,7 @@ paint(const ncplane* p, struct crender* rvec, int dstleny, int dstlenx,
         // if the following is true, we're a real glyph, and not the right-hand
         // side of a wide glyph (nor the null codepoint).
         if( (targc->gcluster = vis->gcluster) ){ // index copy only
-          crender->blitterstacked = cell_blitted_p(vis);
+          crender->s.blittedquads = cell_blittedquadrants(vis);
           // we can't plop down a wide glyph if the next cell is beyond the
           // screen, nor if we're bisected by a higher plane.
           if(cell_double_wide_p(vis)){
@@ -403,7 +406,7 @@ lock_in_highcontrast(nccell* targc, struct crender* crender){
   if(cell_bg_alpha(targc) == CELL_ALPHA_TRANSPARENT){
     cell_set_bg_default(targc);
   }
-  if(crender->highcontrast){
+  if(crender->s.highcontrast){
     // highcontrast weighs the original at 1/4 and the contrast at 3/4
     if(!cell_fg_default_p(targc)){
       crender->fgblends = 3;
@@ -429,7 +432,7 @@ postpaint_cell(nccell* lastframe, int dimx, struct crender* crender,
   lock_in_highcontrast(targc, crender);
   nccell* prevcell = &lastframe[fbcellidx(y, dimx, *x)];
   if(cellcmp_and_dupfar(pool, prevcell, crender->p, targc) > 0){
-    crender->damaged = true;
+    crender->s.damaged = true;
     assert(!cell_wide_right_p(targc));
     const int width = targc->width;
     for(int i = 1 ; i < width ; ++i){
@@ -443,7 +446,7 @@ postpaint_cell(nccell* lastframe, int dimx, struct crender* crender,
       targc->channels = crender[-i].c.channels;
       targc->stylemask = crender[-i].c.stylemask;
       if(cellcmp_and_dupfar(pool, prevcell, crender->p, targc) > 0){
-        crender->damaged = true;
+        crender->s.damaged = true;
       }
     }
   }
@@ -883,7 +886,7 @@ notcurses_rasterize_inner(notcurses* nc, const ncpile* p, FILE* out){
       const size_t damageidx = innery * nc->lfdimx + innerx;
       unsigned r, g, b, br, bg, bb, palfg, palbg;
       const nccell* srccell = &nc->lastframe[damageidx];
-      if(!rvec[damageidx].damaged){
+      if(!rvec[damageidx].s.damaged){
         // no need to emit a cell; what we rendered appears to already be
         // here. no updates are performed to elision state nor lastframe.
         ++nc->stats.cellelisions;
@@ -1094,7 +1097,7 @@ int notcurses_refresh(notcurses* nc, int* restrict dimy, int* restrict dimx){
   }
   memset(p.crender, 0, count * sizeof(*p.crender));
   for(int i = 0 ; i < count ; ++i){
-    p.crender[i].damaged = true;
+    p.crender[i].s.damaged = true;
   }
   int ret = notcurses_rasterize(nc, &p, nc->rstate.mstreamfp);
   free(p.crender);
@@ -1128,7 +1131,7 @@ int notcurses_render_to_file(notcurses* nc, FILE* fp){
   }
   memset(p.crender, 0, count * sizeof(*p.crender));
   for(int i = 0 ; i < count ; ++i){
-    p.crender[i].damaged = true;
+    p.crender[i].s.damaged = true;
   }
   int ret = raster_and_write(nc, &p, out);
   free(p.crender);

tests/stacking.cpp

@@ -33,16 +33,32 @@ TEST_CASE("Stacking") {
     };
     auto top = ncplane_create(n_, &opts);
     REQUIRE(nullptr != top);
-    CHECK(0 == ncplane_set_fg_rgb(top, 0xffffff));
-    CHECK(0 == ncplane_set_fg_rgb(n_, 0xffffff));
-    CHECK(1 == ncplane_putwc(top, L'\u2580')); // upper half block
-    CHECK(1 == ncplane_putwc(n_, L'\u2584')); // lower half block
+    // create an ncvisual of 2 rows, 1 column, with the top 0xffffff
+    const uint32_t topv[] = {htole(0xffffffff), htole(0)};
+    auto ncv = ncvisual_from_rgba(topv, 2, 4, 1);
+    REQUIRE(nullptr != ncv);
+    struct ncvisual_options vopts = {
+      .n = top, .scaling = NCSCALE_NONE, .y = 0, .x = 0, .begy = 0, .begx = 0,
+      .leny = 2, .lenx = 1, .blitter = NCBLIT_2x1, .flags = 0,
+    };
+    CHECK(top == ncvisual_render(nc_, ncv, &vopts));
+    ncvisual_destroy(ncv);
+
+    // create an ncvisual of 2 rows, 1 column, with the bottom 0xffffff
+    const uint32_t botv[] = {htole(0), htole(0xffffffff)};
+    ncv = ncvisual_from_rgba(botv, 2, 4, 1);
+    REQUIRE(nullptr != ncv);
+    vopts.n = n_;
+    CHECK(n_ == ncvisual_render(nc_, ncv, &vopts));
+    ncvisual_destroy(ncv);
+
     CHECK(0 == notcurses_render(nc_));
     uint64_t channels;
     auto egc = notcurses_at_yx(nc_, 0, 0, nullptr, &channels);
     REQUIRE(nullptr != egc);
-    // ought yield space with white background
-    CHECK(0 == strcmp(" ", egc));
+    // ought yield space with white background FIXME currently just yields
+    // an upper half block
+    CHECK(0 == strcmp("\u2580", egc));
     CHECK(0xffffff == channels_fg_rgb(channels));
     CHECK(0xffffff == channels_bg_rgb(channels));
     ncplane_destroy(top);