In the rapidly evolving landscape of generative AI, crafting the perfect prompt is both an art and a science. Amazon Bedrock's new Advanced Prompt Optimization tool simplifies this process, offering a systematic way to refine prompts across multiple models. Whether you're migrating to a newer model or seeking better accuracy from your current setup, this feature provides a data-driven loop to enhance performance. Here are ten critical things you need to know to leverage this powerful capability effectively.

1. What Is Advanced Prompt Optimization?

Advanced Prompt Optimization is a new tool within Amazon Bedrock that automates the fine-tuning of prompt templates. It uses a metric-driven feedback loop—taking in your original prompts, sample inputs, ground truth answers, and evaluation criteria—to generate optimized versions. The tool then outputs both original and improved prompts along with performance scores, cost estimates, and latency data. This allows you to make informed decisions about model selection and prompt design without manual trial and error.

2. How the Optimization Loop Works

The optimization process begins when you provide a prompt template, example user inputs, ground truth responses, and an evaluation metric. The tool iteratively adjusts the prompt, tests it against the models, and measures performance against the specified metric. This closed-loop system continues until it achieves the best possible result. You can guide the process with an AWS Lambda function, an LLM-as-a-judge rubric, or even a simple natural language description of what you want improved.

3. Compare Up to Five Models Simultaneously

One standout feature is the ability to optimize and compare prompts across up to five models at once. This is invaluable when evaluating different foundation models for your use case. You can include your current model as a baseline and test up to four others, or simply run your current model to see before-and-after improvements. The side-by-side comparison helps you quickly identify which model and prompt combination yields the best accuracy, cost efficiency, and response latency.

4. Ideal for Model Migration

Migrating from one model to another often introduces performance regressions. Advanced Prompt Optimization directly addresses this by allowing you to optimize prompts for your new model while comparing to existing results. You can select your current model as the baseline and test up to four candidate models. The tool ensures that optimized prompts for the new model meet or exceed the performance of your existing prompts, making migration smoother and more reliable.

5. Improve Performance Without Changing Models

If you're satisfied with your current model but want better results, the tool works just as effectively for performance enhancement. Simply select your current model as the only target, and the optimizer will refine your prompts to improve accuracy on underperforming tasks. It can help you identify weaknesses in your existing prompts and systematically improve them without switching models. This is especially useful for complex, multi-step tasks where prompt phrasing significantly impacts outcomes.

6. Multimodal Input Support

The tool extends beyond text by supporting multimodal inputs. You can include images in PNG, JPG, or PDF formats within your prompt templates. This means you can optimize prompts for tasks like document analysis, image captioning, or visual question answering. The optimizer evaluates how well the model handles multimodal content and adjusts the prompt accordingly. This capability broadens the scope of use cases, from customer support ticket analysis to medical imaging interpretation.

7. Flexible Evaluation Metrics

You have multiple options for defining the evaluation metric that guides optimization. You can supply your own AWS Lambda function for custom scoring, use an LLM-as-a-judge rubric where a language model assesses response quality, or provide a short natural language description of the criteria. This flexibility lets you tailor the optimization to your specific needs—whether you prioritize accuracy, relevance, tone, or any other aspect of model output.

8. Detailed Outputs: Scores, Cost, and Latency

Once optimization completes, you receive comprehensive results. The tool outputs the original and final prompt templates along with evaluation scores for each. It also provides cost estimates and latency metrics, enabling you to weigh performance against operational constraints. This data is crucial for making cost-effective decisions when deploying prompts at scale. You can clearly see the trade-offs between prompt improvements and resource usage.

9. Getting Started on the Console

To begin, navigate to the Advanced Prompt Optimization page in the Amazon Bedrock console and choose Create prompt optimization. Select up to five inference models—your current model as a baseline and up to four others, or just your current model for internal improvement. Then upload your prompt templates and supporting data in the required format. The console guides you through each step, making the setup straightforward even for first-time users.

10. Preparing Prompt Templates in JSONL

Your prompts and data must be formatted in JSONL files. Each line is a separate JSON object containing fields like version (fixed value bedrock-2026-05-14), templateId, promptTemplate, optional steeringCriteria, and required evaluationSamples. Each sample includes inputVariables and referenceResp. You can also specify custom evaluation metrics via customLLMJConfig or evaluationMetricLambdaArn. Proper formatting ensures the optimizer can parse your data correctly.

Advanced Prompt Optimization empowers you to systematically improve model performance, whether you're migrating or fine-tuning existing prompts. By leveraging automated iteration and multi-model comparison, you can save time and reduce guesswork. Start experimenting today to unlock the full potential of your generative AI applications.