A social media company wants to use a large language model (LLM) for content moderation. The company wants to evaluate the LLM outputs for bias and potential discrimination against specific groups or individuals. Which data source should the company use to evaluate the LLM outputs with the LEAST administrative effort?
A. User-generated content
B. Moderation logs
C. Content moderation guidelines
D. Benchmark datasets
Explanation:
Benchmark datasets are pre-validated datasets specifically designed to evaluate machine learning models for bias, fairness, and potential discrimination. These datasets are the most efficient tool for assessing an LLM’s performance against known standards with minimal administrative effort.
Option D (Correct): "Benchmark datasets":This is the correct answer because using standardized benchmark datasets allows the company to evaluate model outputs for bias with minimal administrative overhead.
Option A:"User-generated content" is incorrect because it is unstructured and would require significant effort to analyze for bias.
Option B:"Moderation logs" is incorrect because they represent historical data and do not provide a standardized basis for evaluating bias.
Option C:"Content moderation guidelines" is incorrect because they provide qualitative criteria rather than a quantitative basis for evaluation.
AWS AI Practitioner References:
Evaluating AI Models for Bias on AWS:AWS supports using benchmark datasets to assess model fairness and detect potential bias efficiently.
A company wants to use a large language model (LLM) on Amazon Bedrock for sentiment analysis. The company needs the LLM to produce more consistent responses to the same input prompt. Which adjustment to an inference parameter should the company make to meet these requirements?
A. Decrease the temperature value
B. Increase the temperature value
C. Decrease the length of output tokens
D. Increase the maximum generation length
Explanation:
The temperature parameter in a large language model (LLM) controls the randomness of the model's output. A lower temperature value makes the output more deterministic and consistent, meaning that the model is less likely to produce different results for the same input prompt.
Option A (Correct): "Decrease the temperature value":This is the correct answer because lowering the temperature reduces the randomness of the responses, leading to more consistent outputs for the same input.
Option B:
"Increase the temperature value" is incorrect because it would make the output more random and less consistent.
Option C:
"Decrease the length of output tokens" is incorrect as it does not directly affect the consistency of the responses.
Option D:"Increase the maximum generation length" is incorrect because this adjustment affects the output length, not the consistency of the model’s responses.
AWS AI Practitioner References:
Understanding Temperature in Generative AI Models:AWS documentation explains that adjusting the temperature parameter affects the model’s output randomness, with lower values providing more consistent outputs.
A company has installed a security camera. The company uses an ML model to evaluate the security camera footage for potential thefts. The company has discovered that the model disproportionately flags people who are members of a specific ethnic group. Which type of bias is affecting the model output?
A. Measurement bias
B. Sampling bias
C. Observer bias
D. Confirmation bias
Explanation:
Sampling bias is the correct type of bias affecting the model output when it disproportionately flags people from a specific ethnic group.
Sampling Bias:
Occurs when the training data is not representative of the broader population, leading to skewed model outputs.
In this case, if the model disproportionately flags people from a specific ethnic group, it likely indicates that the training data was not adequately balanced or representative.
Why Option B is Correct:
Reflects Data Imbalance:A biased sample in the training data could result in unfair outcomes, such as disproportionately flagging a particular group.
Common Issue in ML Models:Sampling bias is a known problem that can lead to unfair or inaccurate model predictions.
Why Other Options are Incorrect:
A. Measurement bias:Involves errors in data collection or measurement, not sampling.
C. Observer bias:Refers to bias introduced by researchers or data collectors, not the model's output.
D. Confirmation bias:Involves favoring information that confirms existing beliefs, not relevant to model output bias.
Which option is a benefit of ongoing pre-training when fine-tuning a foundation model (FM)?
A. Helps decrease the model's complexity
B. Improves model performance over time
C. Decreases the training time requirement
D. Optimizes model inference time
Explanation:
Ongoing pre-training when fine-tuning a foundation model (FM) improves model performance over time by continuously learning from new data.
Ongoing Pre-Training:
Involves continuously training a model with new data to adapt to changing patterns, enhance generalization, and improve performance on specific tasks.
Helps the model stay updated with the latest data trends and minimize drift over time.
Why Option B is Correct:
Performance Enhancement:Continuously updating the model with new data improves its accuracy and relevance.
Adaptability:Ensures the model adapts to new data distributions or domain-specific nuances.
Why Other Options are Incorrect:
A. Decrease model complexity:Ongoing pre-training typically enhances complexity by learning new patterns, not reducing it.
C. Decreases training time requirement:Ongoing pre-training may increase the time needed for training.
D. Optimizes inference time:Does not directly affect inference time; rather, it affects model performance.
A company wants to use AI to protect its application from threats. The AI solution needs to check if an IP address is from a suspicious source.
Which solution meets these requirements?
A. Build a speech recognition system.
B. Create a natural language processing (NLP) named entity recognition system.
C. Develop an anomaly detection system.
D. Create a fraud forecasting system.
A company needs to choose a model from Amazon Bedrock to use internally. The company must identify a model that generates responses in a style that the company's employees prefer.
What should the company do to meet these requirements?
A. Evaluate the models by using built-in prompt datasets.
B. Evaluate the models by using a human workforce and custom prompt datasets.
C. Use public model leaderboards to identify the model.
D. Use the model InvocationLatency runtime metrics in Amazon CloudWatch when trying models.
An AI practitioner has built a deep learning model to classify the types of materials in images. The AI practitioner now wants to measure the model performance.
Which metric will help the AI practitioner evaluate the performance of the model?
A. Confusion matrix
B. Correlation matrix
C. R2 score
D. Mean squared error (MSE)
A digital devices company wants to predict customer demand for memory hardware. The company does not have coding experience or knowledge of ML algorithms and needs to develop a data-driven predictive model. The company needs to perform analysis on internal data and external data.
Which solution will meet these requirements?
A. Store the data in Amazon S3. Create ML models and demand forecast predictions by using Amazon SageMaker built-in algorithms that use the data from Amazon S3.
B. Import the data into Amazon SageMaker Data Wrangler. Create ML models and demand forecast predictions by using SageMaker built-in algorithms.
C. Import the data into Amazon SageMaker Data Wrangler. Build ML models and demand forecast predictions by using an Amazon Personalize Trending-Now recipe.
D. Import the data into Amazon SageMaker Canvas. Build ML models and demand forecast predictions by selecting the values in the data from SageMaker Canvas.
An AI practitioner has a database of animal photos. The AI practitioner wants to automatically identify and categorize the animals in the photos without manual human effort.
Which strategy meets these requirements?
A. Object detection
B. Anomaly detection
C. Named entity recognition
D. Inpainting
Explanation:
Object detection is the correct strategy for automatically identifying and categorizing animals in photos.
Object Detection:
A computer vision technique that identifies and locates objects within an image and assigns them to predefined categories.
Ideal for tasks such as identifying animals in photos, where the goal is to detect specific objects (animals) and categorize them accordingly.
Why Option A is Correct:
Automatic Identification:Object detection models can automatically identify different types of animals in the images without manual intervention.
Categorization Capability:Assigns labels to detected objects, fulfilling the requirement for categorizing animals.
Why Other Options are Incorrect:
B. Anomaly detection:Identifies outliers or unusual patterns, not specific objects in images.
C. Named entity recognition:Used in NLP to identify entities in text, not for image processing.
D. Inpainting:Used for filling in missing parts of images, not for detecting or categorizing objects.
A company wants to develop a large language model (LLM) application by using Amazon Bedrock and customer data that is uploaded to Amazon S3. The company's security policy states that each team can access data for only the team's own customers.
Which solution will meet these requirements?
A. Create an Amazon Bedrock custom service role for each team that has access to only the team's customer data.
B. Create a custom service role that has Amazon S3 access. Ask teams to specify the customer name on each Amazon Bedrock request.
C. Redact personal data in Amazon S3. Update the S3 bucket policy to allow team access to customer data.
D. Create one Amazon Bedrock role that has full Amazon S3 access. Create IAM roles for each team that have access to only each team's customer folders.
A company is building a chatbot to improve user experience. The company is using a large language model (LLM) from Amazon Bedrock for intent detection. The company wants to use few-shot learning to improve intent detection accuracy.
Which additional data does the company need to meet these requirements?
A. Pairs of chatbot responses and correct user intents
B. Pairs of user messages and correct chatbot responses
C. Pairs of user messages and correct user intents
D. Pairs of user intents and correct chatbot responses
A medical company deployed a disease detection model on Amazon Bedrock. To comply with privacy policies, the company wants to prevent the model from including personal patient information in its responses. The company also wants to receive notification when policy violations occur.
Which solution meets these requirements?
A. Use Amazon Macie to scan the model's output for sensitive data and set up alerts for potential violations.
B. Configure AWS CloudTrail to monitor the model's responses and create alerts for any detected personal information.
C. Use Guardrails for Amazon Bedrock to filter content. Set up Amazon CloudWatch alarms for notification of policy violations.
D. Implement Amazon SageMaker Model Monitor to detect data drift and receive alerts when model quality degrades.
| Page 1 out of 12 Pages |