How to Prepare Your Company Data for AI

Your team just turned on an AI assistant that can search company files. On day one, someone asks it about the holiday schedule and gets the 2022 version. Someone else asks about pricing and gets a draft proposal that was never approved. A third person, out of curiosity, asks about salaries, and the AI finds a spreadsheet that was shared with “everyone” years ago.

None of that is the AI’s fault. It found exactly what your data allowed it to find. AI tools are only as good, and as safe, as the information you give them. This guide walks through a practical process to get your data ready, whether you’re rolling out an AI assistant, building a chatbot on your documents or training a model.

Why data preparation decides AI success

Most business AI projects use company data in 1 of 3 ways:

• Search and answer assistants (such as Microsoft 365 Copilot, Google Gemini for Workspace or a custom chatbot) that retrieve your documents and summarize them. These usually rely on retrieval-augmented generation.
• Analytics and prediction that use structured data, such as sales records, to forecast or score.
• Fine-tuning or training a model on your examples, which is less common for small and midsize companies. See when to fine-tune an LLM.

All 3 run into the same problems: outdated documents, conflicting versions, missing fields, unclear ownership and loose permissions. Retrieval-based assistants are especially sensitive to permissions. Microsoft’s documentation says Copilot and agents retrieve data through Microsoft Graph and respect existing permissions and sharing settings. That’s good design, but it also means that anything already overshared becomes easy to find.

Key takeaway: AI doesn’t fix messy data; it exposes it. Cleaning, permissioning and documenting your data is the cheapest insurance you can buy before any AI rollout.

Step 1: Start from a use case, not a data lake

Don’t try to clean everything. Pick 1 specific AI use case and prepare only the data it needs. “Answer employee HR questions” needs the handbook, policies and benefits documents. “Predict which invoices will be paid late” needs invoice history and customer payment records.

Write down:

• The question or task the AI will handle
• Who will use it
• What a correct answer looks like
• Which systems hold the data it needs

If you haven’t chosen a project yet, our guide on picking your first AI pilot project helps you score options, and the AI readiness checklist covers the wider picture.

Step 2: Find and inventory the data

List every source the use case touches. For each one, record the system, the owner, what’s in it, how current it is and whether it contains sensitive information.

Knowing whether data is structured (tables) or unstructured (documents, emails, chats) matters, because each needs different preparation. The cleanup steps below cover both.

Step 3: Clean and curate

For documents, curation matters more than cleaning:

• Remove or archive outdated content. Old policies and superseded price lists are the top cause of wrong AI answers.
• Pick 1 source of truth for each topic and archive duplicates.
• Add clear titles and dates so both people and AI can tell what’s current.
• Convert scanned images to searchable text where the tool needs to read them.
• Fix structure. Clear headings and short sections make documents easier for retrieval systems to split and match.

For structured data:

• Remove duplicate records
• Standardize formats for dates, currencies, names and categories
• Fill or flag missing values
• Fix obvious errors and outliers
• Make sure definitions are consistent, for example what counts as an “active customer”

Our guide on how to improve data quality goes deeper on both.

Step 4: Fix permissions before the AI does it for you

This is the step companies most often skip, and the one that causes the most embarrassing incidents. If an AI assistant searches on behalf of each user, it can surface anything that user technically has access to, even files they never knew existed.

Microsoft’s guidance on preparing for Copilot with SharePoint Advanced Management includes these actions, which are good practice on any platform:

1. Review default sharing settings. Microsoft notes that SharePoint sets sharing to the most permissive option by default.
2. Run access reports to find sites and files shared with everyone in the organization or through “Anyone” links.
3. Restrict access to sensitive sites to specific security groups.
4. Exclude high-risk content from AI discovery where it must stay accessible but shouldn’t be broadly searchable.
5. Clean up ownerless and inactive sites, and archive what nobody uses.

Google Workspace, Box, Dropbox and other platforms have their own sharing reports and controls. The principle is the same: least privilege first, then turn on AI. Our guide to identity and access management explains the model.

Step 5: Protect sensitive and regulated data

Before data goes into any AI tool, decide what should never be included or must be handled specially:

• Personal information about customers and employees
• Health, financial or children’s data, which may be covered by laws such as HIPAA, GLBA or COPPA, depending on your business
• Confidential business information such as contracts, trade secrets and M&A documents
• Credentials and secrets that sometimes hide in documents and tickets

Options include removing the data, masking or redacting it, labeling it with sensitivity tags so tools can respect them, or keeping that data source out of the AI project entirely. Check your vendor’s data use and retention terms too; our guide on whether it’s safe to put company data into AI lists the questions to ask. Also align with your data retention policy, since data you no longer need is a liability, not an asset.

This is not legal advice. If you handle regulated data, check your obligations with counsel before connecting it to AI tools.

Step 6: Document each dataset

Documentation is what lets you trust, audit and fix an AI system later. A widely cited research paper, “Datasheets for Datasets” (Gebru and colleagues, published in Communications of the ACM in 2021), proposed that every dataset come with a document describing its motivation, composition, collection process and recommended uses. You don’t need an academic template. A 1-page record per dataset works:

• Name and owner
• Source systems and how the data was collected
• What’s included and excluded, including date ranges
• Known gaps, errors or biases
• Sensitivity level and who may access it
• Approved uses, and uses that are off-limits
• Update schedule and who maintains it

NIST’s AI Risk Management Framework, a voluntary framework released in January 2023, encourages this kind of documentation as part of managing AI risk across the life cycle. If you’re setting up broader rules, see what data governance is.

Where to start: a 30-day plan

1. Week 1: Choose 1 use case. List the data sources it needs and name an owner for each.
2. Week 2: Archive outdated and duplicate documents. Fix the most important data quality issues in structured sources.
3. Week 3: Run sharing and permission reports. Close broad access on sensitive content and decide what stays out of the AI project.
4. Week 4: Write a 1-page datasheet for each source. Build a small set of test questions with known correct answers, then pilot the AI tool with a small group and check its answers against your test set.

After launch, assign someone to keep the content current. A quarterly review of new documents, permissions and test questions keeps answers accurate as your business changes.

Conclusion

Preparing data for AI isn’t glamorous, but it’s where most of the value and most of the risk live. Start from 1 use case, find the data it needs, clean and curate it, lock down permissions, protect sensitive information and document what you have. Do that well once, and every AI project after it gets faster, safer and more accurate.

Sources

• Microsoft Learn: Get ready for Microsoft Copilot with SharePoint Advanced Management — Copilot respects existing permissions, default sharing settings, access reports and restricted content discovery
• NIST: AI Risk Management Framework — voluntary framework released January 26, 2023, and the Generative AI Profile
• Gebru et al., “Datasheets for Datasets” (arXiv preprint of the Communications of the ACM paper) — documenting a dataset’s motivation, composition, collection process and recommended uses