Skip to content

Create and Run a Crawler

A crawler automatically connects to your data store and discovers its datasets.

To create and run a crawler in the UI:

  1. Log into the Okera UI as a user with permissions to create crawlers. See Who Has Access to Crawlers.

  2. Select Register on the side menu to access the Registration page. All the crawlers created in the Okera environment are listed.

  3. Select on the Registration page. The Create crawler dialog appears.

    Create crawler

  4. Supply the following values on the dialog:

    • Use the drop-down list in the Source connection box to select the Okera connection to the data store you wish to crawl. For more information about connections see Connection Overview.

    • For relational data store connections, use the drop-down lists in the Source database and Source schema boxes to select the database and schema to be crawled.

      For object storage connections, specify the path for the bucket or folder to be crawled. For more information about configuring crawlers for object storage connections, see Configure Object Storage Crawlers.

    • Specify a unique name for your crawler in the Crawler name box.

  5. Select on the modal. The crawler is saved and runs. It appears in the list of crawlers on the Registration page with the status Crawling… while it is running. When the crawler finishes running, you can register the datasets it discovered to Okera databases for policy enforcement. See Register Datasets.

See Crawler Questions and Answers to diagnose any common errors that occur during crawler creation and processing.