{"templateId":"markdown","sharedDataIds":{"sidebar":"sidebar-v2/sidebars.yaml"},"props":{"metadata":{"markdoc":{"tagList":[]},"type":"markdown"},"seo":{"title":"Bulk Enrich Companies"},"dynamicMarkdocComponents":[],"compilationErrors":[],"ast":{"$$mdtype":"Tag","name":"article","attributes":{},"children":[{"$$mdtype":"Tag","name":"Heading","attributes":{"level":1,"id":"bulk-enrich-companies","__idx":0},"children":["Bulk Enrich Companies"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["This guide covers how to efficiently enrich large datasets — from thousands to hundreds of thousands of companies — using the HG API."]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":2,"id":"how-it-works","__idx":1},"children":["How It Works"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["The ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["/companies/enrich"]}," endpoint accepts up to 25 companies per request. Your throughput depends on your plan's rate limit — see ",{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"/v2/guides/rate-limiting"},"children":["Rate Limiting"]},". On the Standard plan (25 requests/second), that's up to ",{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["625 companies per second"]},", or roughly ",{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["100,000 companies in under 3 minutes"]},"."]},{"$$mdtype":"Tag","name":"div","attributes":{"className":"md-table-wrapper"},"children":[{"$$mdtype":"Tag","name":"table","attributes":{"className":"md"},"children":[{"$$mdtype":"Tag","name":"thead","attributes":{},"children":[{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"th","attributes":{"data-label":"Dataset Size"},"children":["Dataset Size"]},{"$$mdtype":"Tag","name":"th","attributes":{"data-label":"Estimated Time (Standard plan)"},"children":["Estimated Time (Standard plan)"]}]}]},{"$$mdtype":"Tag","name":"tbody","attributes":{},"children":[{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["1,000 companies"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["~2 seconds"]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["10,000 companies"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["~16 seconds"]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["100,000 companies"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["~3 minutes"]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["500,000 companies"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["~14 minutes"]}]}]}]}]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":2,"id":"before-you-start","__idx":2},"children":["Before You Start"]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":3,"id":"1-check-your-credit-balance","__idx":3},"children":["1. Check your credit balance"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["Large enrichments consume significant credits. Use the Credits API to verify you have enough before starting:"]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"data-language":"bash","header":{"controls":{"copy":{}}},"source":"curl https://api.hginsights.com/data-api/v2/credits \\\n  -H \"Authorization: Bearer $HG_API_KEY\"\n","lang":"bash"},"children":[]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":3,"id":"2-only-request-what-you-need","__idx":4},"children":["2. Only request what you need"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["Each field group adds to your credit consumption. If you only need firmographics, don't request technographics and spend:"]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"data-language":"json","header":{"controls":{"copy":{}}},"source":"{\n  \"companies\": {\"domains\": [\"walmart.com\"]},\n  \"fields\": [\"firmographics\"]\n}\n","lang":"json"},"children":[]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":3,"id":"3-use-domains-or-hg-ids","__idx":5},"children":["3. Use domains or HG IDs"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["If you have company names but no domains or IDs, use the ",{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["Company Match"]}," endpoint first to resolve them, then enrich using the returned IDs."]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":2,"id":"implementation","__idx":6},"children":["Implementation"]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":3,"id":"step-1-split-into-batches-of-25","__idx":7},"children":["Step 1: Split into batches of 25"]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"data-language":"python","header":{"controls":{"copy":{}}},"source":"import requests\n\nAPI_KEY = \"your_api_key\"\nBASE_URL = \"https://api.hginsights.com/data-api/v2/companies/enrich\"\nHEADERS = {\n    \"Authorization\": f\"Bearer {API_KEY}\",\n    \"Content-Type\": \"application/json\"\n}\n\ndomains = [\"walmart.com\", \"google.com\", ...]  # your full list\n\n# Split into chunks of 25\nbatches = [domains[i:i+25] for i in range(0, len(domains), 25)]\n","lang":"python"},"children":[]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":3,"id":"step-2-send-requests-with-rate-limiting","__idx":8},"children":["Step 2: Send requests with rate limiting"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["Stay within the 25 requests/second limit. A simple approach is to add a small delay between requests:"]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"data-language":"python","header":{"controls":{"copy":{}}},"source":"import time\n\nresults = []\n\nfor batch in batches:\n    payload = {\n        \"companies\": {\"domains\": batch},\n        \"fields\": [\"firmographics\", \"technographics\", \"spend\"]\n    }\n\n    response = requests.post(BASE_URL, json=payload, headers=HEADERS)\n\n    if response.status_code == 200:\n        results.extend(response.json()[\"companies\"])\n    elif response.status_code == 429:\n        # Rate limited — wait and retry\n        time.sleep(2)\n        response = requests.post(BASE_URL, json=payload, headers=HEADERS)\n        results.extend(response.json()[\"companies\"])\n\n    time.sleep(0.05)  # ~20 requests/second, safely under the limit\n","lang":"python"},"children":[]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":3,"id":"step-3-handle-rate-limits-gracefully","__idx":9},"children":["Step 3: Handle rate limits gracefully"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["If you hit a 429 (Too Many Requests) response, back off and retry:"]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"data-language":"python","header":{"controls":{"copy":{}}},"source":"def enrich_with_retry(payload, max_retries=3):\n    for attempt in range(max_retries):\n        response = requests.post(BASE_URL, json=payload, headers=HEADERS)\n\n        if response.status_code == 200:\n            return response.json()[\"companies\"]\n        elif response.status_code == 429:\n            wait_time = 2 ** attempt  # 1s, 2s, 4s\n            time.sleep(wait_time)\n        else:\n            response.raise_for_status()\n\n    raise Exception(\"Max retries exceeded\")\n","lang":"python"},"children":[]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":2,"id":"parallel-requests","__idx":10},"children":["Parallel Requests"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["For maximum throughput, send requests in parallel using a thread pool. This example sends up to 20 concurrent requests:"]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"data-language":"python","header":{"controls":{"copy":{}}},"source":"from concurrent.futures import ThreadPoolExecutor, as_completed\nimport time\n\ndef enrich_batch(batch):\n    payload = {\n        \"companies\": {\"domains\": batch},\n        \"fields\": [\"firmographics\", \"technographics\"]\n    }\n    response = requests.post(BASE_URL, json=payload, headers=HEADERS)\n    if response.status_code == 429:\n        time.sleep(2)\n        response = requests.post(BASE_URL, json=payload, headers=HEADERS)\n    response.raise_for_status()\n    return response.json()[\"companies\"]\n\nresults = []\nwith ThreadPoolExecutor(max_workers=20) as executor:\n    futures = {executor.submit(enrich_batch, batch): batch for batch in batches}\n    for future in as_completed(futures):\n        results.extend(future.result())\n","lang":"python"},"children":[]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":2,"id":"handling-large-technographics-responses","__idx":11},"children":["Handling Large Technographics Responses"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["Some companies have thousands of technology installs. The API paginates technographics results — use the ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["pagination"]}," parameter to retrieve all installs:"]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"data-language":"json","header":{"controls":{"copy":{}}},"source":"{\n  \"companies\": {\"domains\": [\"walmart.com\"]},\n  \"fields\": [\"technographics\"],\n  \"pagination\": {\n    \"technographics\": {\"limit\": 100, \"offset\": 0}\n  }\n}\n","lang":"json"},"children":[]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["Increment ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["offset"]}," by ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["limit"]}," until you've retrieved all installs (check ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["installs_count"]}," in the response)."]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":2,"id":"tips","__idx":12},"children":["Tips"]},{"$$mdtype":"Tag","name":"ul","attributes":{},"children":[{"$$mdtype":"Tag","name":"li","attributes":{},"children":[{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["Start small"]}," — Test with 100 companies first to validate your pipeline and estimate credit usage"]},{"$$mdtype":"Tag","name":"li","attributes":{},"children":[{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["Log failures"]}," — Track which batches fail so you can retry them without re-processing the entire dataset"]},{"$$mdtype":"Tag","name":"li","attributes":{},"children":[{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["Store results"]}," — Write results to a database or file as you go, rather than holding everything in memory"]},{"$$mdtype":"Tag","name":"li","attributes":{},"children":[{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["Filter technographics"]}," — If you only need specific vendors or products, use filters to reduce response size and credit consumption"]}]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":2,"id":"need-recurring-large-scale-data","__idx":13},"children":["Need Recurring Large-Scale Data?"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["If you're enriching large datasets on a regular basis, the API may not be the most cost-effective option. HG Insights also offers bulk data deliveries tailored to the companies you care about. Contact your account manager to learn about dataset licensing options."]}]},"headings":[{"value":"Bulk Enrich Companies","id":"bulk-enrich-companies","depth":1},{"value":"How It Works","id":"how-it-works","depth":2},{"value":"Before You Start","id":"before-you-start","depth":2},{"value":"1. Check your credit balance","id":"1-check-your-credit-balance","depth":3},{"value":"2. Only request what you need","id":"2-only-request-what-you-need","depth":3},{"value":"3. Use domains or HG IDs","id":"3-use-domains-or-hg-ids","depth":3},{"value":"Implementation","id":"implementation","depth":2},{"value":"Step 1: Split into batches of 25","id":"step-1-split-into-batches-of-25","depth":3},{"value":"Step 2: Send requests with rate limiting","id":"step-2-send-requests-with-rate-limiting","depth":3},{"value":"Step 3: Handle rate limits gracefully","id":"step-3-handle-rate-limits-gracefully","depth":3},{"value":"Parallel Requests","id":"parallel-requests","depth":2},{"value":"Handling Large Technographics Responses","id":"handling-large-technographics-responses","depth":2},{"value":"Tips","id":"tips","depth":2},{"value":"Need Recurring Large-Scale Data?","id":"need-recurring-large-scale-data","depth":2}],"frontmatter":{"seo":{"title":"Bulk Enrich Companies"}},"lastModified":"2026-09-24T20:33:24.000Z","pagePropGetterError":{"message":"","name":""}},"slug":"/v2/guides/bulk-enrichment","userData":{"isAuthenticated":false,"teams":["anonymous"]},"isPublic":true}