---
title: SDXL
---

import { Image } from 'astro:assets';
import maskImage from '../../../assets/sdxl-mask-image.png';
import originalImage from '../../../assets/sdxl-inpainting-original-image.png';
import inpaintedImage from '../../../assets/sdxl-inpainting-inpainted-image.png';
import icon from '../../../assets/icon-stability.svg';

<span title="Label: Pro" data-view-component="true" class="Label Label--api text-uppercase">
  Unstable API
</span>
<span title="Label: Pro" data-view-component="true" class="Label Label--version text-uppercase">
  0.8.0
</span>
<span title="Label: Pro" data-view-component="true" class="Label Label--package">
  <a target="_blank" href="https://www.npmjs.com/package/@project-lakechain/bedrock-image-generators">
    @project-lakechain/bedrock-image-generators
  </a>
</span>
<span class="language-icon">
  <svg role="img" viewBox="0 0 24 24" width="30" xmlns="http://www.w3.org/2000/svg" style="fill: #3178C6;"><title>TypeScript</title><path d="M1.125 0C.502 0 0 .502 0 1.125v21.75C0 23.498.502 24 1.125 24h21.75c.623 0 1.125-.502 1.125-1.125V1.125C24 .502 23.498 0 22.875 0zm17.363 9.75c.612 0 1.154.037 1.627.111a6.38 6.38 0 0 1 1.306.34v2.458a3.95 3.95 0 0 0-.643-.361 5.093 5.093 0 0 0-.717-.26 5.453 5.453 0 0 0-1.426-.2c-.3 0-.573.028-.819.086a2.1 2.1 0 0 0-.623.242c-.17.104-.3.229-.393.374a.888.888 0 0 0-.14.49c0 .196.053.373.156.529.104.156.252.304.443.444s.423.276.696.41c.273.135.582.274.926.416.47.197.892.407 1.266.628.374.222.695.473.963.753.268.279.472.598.614.957.142.359.214.776.214 1.253 0 .657-.125 1.21-.373 1.656a3.033 3.033 0 0 1-1.012 1.085 4.38 4.38 0 0 1-1.487.596c-.566.12-1.163.18-1.79.18a9.916 9.916 0 0 1-1.84-.164 5.544 5.544 0 0 1-1.512-.493v-2.63a5.033 5.033 0 0 0 3.237 1.2c.333 0 .624-.03.872-.09.249-.06.456-.144.623-.25.166-.108.29-.234.373-.38a1.023 1.023 0 0 0-.074-1.089 2.12 2.12 0 0 0-.537-.5 5.597 5.597 0 0 0-.807-.444 27.72 27.72 0 0 0-1.007-.436c-.918-.383-1.602-.852-2.053-1.405-.45-.553-.676-1.222-.676-2.005 0-.614.123-1.141.369-1.582.246-.441.58-.804 1.004-1.089a4.494 4.494 0 0 1 1.47-.629 7.536 7.536 0 0 1 1.77-.201zm-15.113.188h9.563v2.166H9.506v9.646H6.789v-9.646H3.375z"/></svg>
</span>
<span class="language-icon" style="margin-right: 10px">
  <a target="_blank" href="https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-stability-diffusion.html">
    <Image width="31" src={icon} alt="Icon" style="border-radius: 50%" />
  </a>
</span>
<div style="margin-top: 26px"></div>

---

The Bedrock image generators package enables developers to use image generation models hosted on [Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html), and create images at scale within their pipelines.
The Stable Diffusion XL image generator is part of this package and we showcase in this documentation how to leverage all of its functionalities.

---

### 🖼️ Text-to-Image

To generate images as part of your pipelines from a text prompt, you can use the `SdxlImageGenerator` construct.
This construct either takes the content of input text document as a prompt for generating an image, allowing you to nicely chain documents together,
or can also take a static prompt.

> ℹ️ The below example demonstrates how to use the SDXL image generator to create images from a static prompt every 5 minutes, using the [Scheduler Trigger](/project-lakechain/triggers/scheduler-event-trigger).

#### Example

```typescript
import * as cdk from '@aws-cdk/core';
import * as scheduler from '@aws-cdk/aws-scheduler-alpha';
import { SchedulerEventTrigger } from '@project-lakechain/scheduler-event-trigger';
import { SdxlImageGenerator } from '@project-lakechain/bedrock-image-generators';

class Stack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string) {
    // The scheduler trigger.
    const trigger = new SchedulerEventTrigger.Builder()
      .withScope(this)
      .withIdentifier('Trigger')
      .withScheduleExpression(
        scheduler.ScheduleExpression.rate(cdk.Duration.minutes(5))
      )
      .build();

    // The SDXL image generator.
    const imageGenerator = new SdxlImageGenerator.Builder()
      .withScope(this)
      .withIdentifier('SdxlImageGenerator')
      .withSource(trigger)
      .withPrompt(`A beautiful sunset over the ocean.`)
      .withNegativePrompts([
        'low resolution',
        'low quality'
      ])
      .build();
  }
}
```

<br />
<br />

---


### 👉 Using References

While generating images based on a static prompt can be useful, the true power of Lakechain lies in its ability to combine
multiple middlewares together to process documents dynamically. To generate images based on previous documents, you can use Lakechain references.

References allows to reference the document currently processed by a middleware at runtime.

<br />

---

#### Content Reference

Below is an example where we dynamically reference the *content* of a text document as the prompt given to SDXL for generating an image.

```typescript
import { SdxlImageGenerator } from '@project-lakechain/bedrock-image-generators';
import * as r from '@project-lakechain/core/dsl/vocabulary/reference';

const imageGenerator = new SdxlImageGenerator.Builder()
  .withScope(this)
  .withIdentifier('SdxlImageGenerator')
  .withSource(trigger)
  .withPrompt(r.reference(r.document())) // 👈 Reference
  .build();
```

<br />

---

#### Attribute Reference

You can also reference an *attribute* of the document [Cloud Event](/project-lakechain/general/events) as the prompt.
For example, using the document `description` attribute as a prompt to generate an image representing the current document.

```typescript
import { SdxlImageGenerator } from '@project-lakechain/bedrock-image-generators';
import * as r from '@project-lakechain/core/dsl/vocabulary/reference';

const imageGenerator = new SdxlImageGenerator.Builder()
  .withScope(this)
  .withIdentifier('SdxlImageGenerator')
  .withSource(trigger)
  .withPrompt(r.reference(
    r.attribute('data.metadata.description') // 👈 Reference
  ))
  .build();
```

<br />

---

### 🧑‍🎨 Image Inpainting

SDXL models on Amazon Bedrock also support image inpainting, which refers to the process of painting a new image within an existing image.
SDXL supports generating an image and "painting" it within an existing image using a mask image, delimiting the area where the new image should be painted.

<p align="center">
  <table style="display: table; margin: auto">
    <tr>
      <th>Original Image</th>
      <th>Inpainted Image</th>
    </tr>
    <tr>
      <td style="padding-top: 0.8em; padding-bottom: 1em">
        <Image width="200" src={originalImage} alt="Original Image" />
      </td>
      <td style="padding-top: 0.8em; padding-bottom: 1em">
        <Image width="190" src={inpaintedImage} alt="Inpainted Image" />
      </td>
    </tr>
  </table>
</p>

The area for inpainting is defined using a black mask image, outlining the region of the image that should be inpainted.
Below is an example of a mask we are using to inpaint the above image.

<p align="center">
  <Image src={maskImage} alt="Mask Image" width="300" />
</p>

To implement inpainting within a pipeline, you can use the `SdxlImageGenerator` by specifying a source image
as well as a mask image. We leverage *Lakechain references* to specify the source and mask images.

> 💁 In the below example we create a pipeline that triggers when an image is uploaded to a source S3 bucket,
this image will be used as the base for inpainting.

```typescript
import { S3EventTrigger } from '@project-lakechain/s3-event-trigger';
import { SdxlImageGenerator } from '@project-lakechain/bedrock-image-generators';
import { CacheStorage } from '@project-lakechain/core';
import * as r from '@project-lakechain/core/dsl/vocabulary/reference';

class Stack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string) {
    const cache = new CacheStorage(this, 'Cache');

    // Create the S3 event trigger.
    const trigger = new S3EventTrigger.Builder()
      .withScope(this)
      .withIdentifier('Trigger')
      .withCacheStorage(cache)
      .withBucket(bucket)
      .build();
    
    // Modify the input images using SDXL on Amazon Bedrock.
    const imageGenerator = new SdxlImageGenerator.Builder()
      .withScope(this)
      .withIdentifier('ImageGenerator')
      .withCacheStorage(cache)
      .withSource(source)
      // Specify a prompt for the inpainted image.
      .withPrompt('A glowing red cloud')
      .withModelParameters({
        // We reference the input document as the `init_image` parameter.
        init_image: r.reference(r.document()),
        // We reference the mask image as the `mask_image` parameter.
        mask_image: r.reference(
          r.url(`s3://${source.bucketName}/mask.png`)
        ),
        mask_source: 'MASK_IMAGE_BLACK'
      })
      .build();
  }
}
```

> ℹ️ This example assumes that the mask image exists in the source bucket.

<br />
<br />

---

### 🌐 Region Selection

You can specify the AWS region in which you want to invoke Amazon Bedrock using the `.withRegion` API.

> 💁 By default, the middleware will use the current region in which it is deployed.

```typescript
import { SdxlImageGenerator, SdxlImageModel } from '@project-lakechain/bedrock-image-generators';

const imageGenerator = new SdxlImageGenerator.Builder()
  .withScope(this)
  .withIdentifier('SdxlImageGenerator')
  .withSource(trigger)
  .withRegion('us-east-1') // 👈 Alternate region
  .withImageModel(SdxlImageModel.STABILITY_DIFFUSION_XL_V0)
  .build();
```

<br />
<br />

---

### ⚙️ Parameters

You can pass additional parameters to the SDXL model for customizing how the SDXL model generates images.
All the parameters supported by SDXL are supported.

> 💁 You can find a list of supported parameters in the [AWS SDXL model documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-diffusion-1-0-text-image.html).

```typescript
import { SdxlImageGenerator } from '@project-lakechain/bedrock-image-generators';

const imageGenerator = new SdxlImageGenerator.Builder()
  .withScope(this)
  .withIdentifier('SdxlImageGenerator')
  .withSource(trigger)
  .withModelParameters({
    cfg_scale: 5,
    steps: 20
  })
  .build();
```

<br />
<br />

---

### 🏗️ Architecture

This middleware is based on a Lambda compute and interacts with the Amazon Bedrock service in the specified
region to generate images.

![Architecture](../../../assets/sdxl-image-generator-architecture.png)

<br />
<br />

---

### 🏷️ Properties

<br />

##### Supported Inputs

|  Mime Type  | Description |
| ----------- | ----------- |
| `text/plain` | Text document |
| `image/png` | Image document |
| `image/jpeg` | Image document | 
| `application/json+scheduler` | Scheduler event |

##### Supported Outputs

|  Mime Type  | Description |
| ----------- | ----------- |
| `image/png` | Image document |

##### Supported Compute Types

| Type  | Description |
| ----- | ----------- |
| `CPU` | This middleware only supports CPU compute. |

<br />
<br />

---

### 📖 Examples

- [Image Generation Pipeline](https://github.com/awslabs/project-lakechain/tree/main/examples/simple-pipelines/generative-pipelines/image-generation-pipeline) - An example showcasing how to generate image using Amazon Bedrock models.
- [Image Article Pipeline](https://github.com/awslabs/project-lakechain/tree/main/examples/simple-pipelines/generative-pipelines/image-article-pipeline) - An example showcasing how to generate images from articles using Amazon Bedrock.
