Catalog import and maintenance

Learn how to easily bulk onboard and apply changes to your catalog

Catalog import feature is a way of bulk onboarding your services using one service as template and pulling information from Repositories, Cloud Resources or Users/Teams to generate a tsv list. That tsv can be manually or programmatically filled with your information and then imported back to configure8 where we will create and map all that information for you.

Summary - how to use Catalog Onboarding feature to onboard your catalog

1. Connect credentials

2. Create a fully configured service in the configure8 UI.

3. Create and download the tsv

4. Edit and enhance the tsv with your data sources

5. Upload the tsv back into configure8

6. Done! We just created your full catalog of services with all the info in the tsv

The following diagram illustrates the above steps:

When generating the tsv, the service source will be used to create the rows and all information added to the template service will create the columns of the tsv files

Connect credentials

To configure your template service you need to have the credentials created for the tools your organization uses. See the full list and instructions for credentials and plugins supported by configure8 here.

Create a full configured service

Create a service that represents what you want to see in your service catalog, adding plugins from the credentials you created in last section, as well as useful links and metadata.

This is an good example of a fully configured service where you can see repository, CI/CD, plugins added to the service page, owners, metadata and links:

Create tsv file

To create the tsv go to Settings -> Catalog Import and click Generate TSV Template button.

Search and select for the template service name and select what will be the source of the information for generating the rows of your tsv file.

The following sources are supported at this time:

• Repository: lists all repositories connected with the repository provider credentials. Recommended in case you want one repository for each service.

• Resource: use cloud resources to pull information. Useful if you have a k8s cluster namespace for each service, for example.

• Team: list all teams in configure8. Recommended in case you have a 1:1 team to service mapping. You can create teams via API, or activating SSO and SCIM integration and we will pull that data from your Identity Provider.

• Person: same as teams above.

Additionally, filters will be provided depending on the data available in your catalog and credentials. For Repository as data source, we will display different providers as filters if those are present:

Click on Download.

Edit and enhance tsv file

You can import the tsv file in Excel, Numbers or Google sheets and bulk add information to it. You can also programmatically add information to it via a script or a simple program.

We recommend hand coding the import file, or using Numbers or Excel, due to the way Google Sheets handles escaping fields with quotes. Its possible to use Google sheets but you have to manually re-add the quotes for arrays fields.

The tsv file will follow this structure:

• First row: headers

• Second row: example information from the template service. In this row you can see the expected format for the fields.

• Third row and on: this are the lines from the sources and filters you selected in the previous step. The only thing we add here is the index needed for our system and the Name that is being infered from the source.

Below is a sample TSV file and the expected behavior for each field added described on the third row under parenthesis ().

From ID	Name	Description	Repository	Applications	Environments	Links	Metadata	Owners	Viewers	Plugin 0 [sonarcloud_repo_profile] Credential ID	Plugin 0 [sonarcloud_repo_profile] Credential Configuration	Plugin 1 [pagerduty_oncall] Credential ID	Plugin 1 [pagerduty_oncall] Credential Configuration	Plugin 2 [jira_activity] Credential ID	Plugin 2 [jira_activity] Credential Configuration	Plugin 4 [datadog_monitor] Credential ID	Plugin 4 [datadog_monitor] Credential Configuration	Plugin 5 [datadog_slo] Credential ID	Plugin 5 [datadog_slo] Credential Configuration	Plugin 7 [github_actions] Credential ID	Plugin 8 [circle_ci] Credential ID	Plugin 8 [circle_ci] Credential Configuration
C8::SERVICE_ID 7f6173ec-b559-4420-93b8-b22cf3e19601	AWS Bookstore Lambda	Lambda functions bookstore implementation	https://github.com/DEMO-c8/aws-bookstore-demo-app	Bookstore	QA, dev, production	[url=https://docs.google.com/document/d/1O5aEc91nnb9Rqu1DW-57JKpNd40KXv0eSBIqTn9Esok/edit,title=Runbook,icon=runbook], [url=https://aws.github.io/code-of-conduct-faq,title=Code of Conduct,icon=documentation]	[type=Text,name=Business Unit,value=007], [type=AutoMap,name=app,value=webbookstore]	[e-mail]	[e-mail]	4ecfb01d-1700-4db2-82b6-ea85c3efe0b1	"[organizationKey=demo-cfg8,project=demo-cfg8_aws-bookstore-demo-app,metrics=""bugs,code_smells,cognitive_complexity,coverage,violations,vulnerabilities,new_bugs,open_issues,ncloc_data""]"	a90bb9b9-f2f6-4711-a5f9-4e0c44da7b15	[schedule_id=PLJ6A5A,service_id=P0RJ7LK]	678e70d5-c398-4cb9-a62b-08b5b9335b81	[project_key=SAN]	e8ec9aa1-1d0e-404b-928e-6c86900f4c0f	"[monitor_ids=""108373045,118484114""]"	e8ec9aa1-1d0e-404b-928e-6c86900f4c0f	"[slo_ids=""c2511ccb189753ffaaf9aba7fa13d639,ff6f1e0740465d6fa4c003ba4b4bbf3a""]"	82fbf4e7-867a-4d77-9c09-13db1b6b5393	3d94cfc5-8b36-49df-bf35-439f3f20bb37	[project_slug=github/demo-cfg8/Azure-demo-APP]
c8455a5a-967e-4eb8-ae92-d1714f894837	(Required - Service name to be created Infered from repository) SSA-DEMO	(Service description) This is the DEMO Self-Serve Actions Service	https://github.com/DEMO-c8/SSA-demo	(Applications that this service is part of. Non existing ones will be created. Service will be added to the applications) SSA APP, Application 2	(Environments this service runs on - Non existing ones will be created.) Production,QA,dev	(Links you want to add to this service) [url=https://docs.google.com/document/d/1O5aEc91nnb9Rqu1DW-57JKpNd40KXv0eSBIqTn9Esok/edit,title=Runbook,icon=runbook], [url=https://aws.github.io/code-of-conduct-faq,title=Code of Conduct,icon=documentation]	(Metadata you want to add to this service) [type=Text,name=Business Unit,value=008	(Owners of the service. valid emails that don't exist in configure8 will be invited. Strings will create groups with that name) [e-mail]	(Viewers of the service. valid emails) [e-mail]	(required - Credential for this plugin. If not present this service will not be created) 4ecfb01d-1700-4db2-82b6-ea85c3efe0b1		(required - Credential for this plugin. If not present this service will not be created) a90bb9b9-f2f6-4711-a5f9-4e0c44da7b15		(required - Credential for this plugin. If not present this service will not be created) 678e70d5-c398-4cb9-a62b-08b5b9335b81	(required - Project key for Jira plugin. If not present this service will not be created) [project_key=SAN]	(required - Credential for this plugin. If not present this service will not be created) e8ec9aa1-1d0e-404b-928e-6c86900f4c0f	(required - List of monitors for Datadog Plugin. You can use a default monitor if you don't have that info for all services and ask owners to update later) "[monitor_ids=""108373045,118484114""]"	(required - Credential for this plugin. If not present this service will not be created) e8ec9aa1-1d0e-404b-928e-6c86900f4c0f	(required - List of SLO's for Datadog Plugin. You can use a default monitor if you don't have that info for all services and ask owners to update later) "[slo_ids=""c2511ccb189753ffaaf9aba7fa13d639,ff6f1e0740465d6fa4c003ba4b4bbf3a""]"	(required - Credential for this plugin. If not present this service will not be created) 82fbf4e7-867a-4d77-9c09-13db1b6b5393	(required - Credential for this plugin. If not present this service will not be created) 3d94cfc5-8b36-49df-bf35-439f3f20bb37	(required - Project for CCI Plugin. You can use a default monitor if you don't have that info for all services and ask owners to update later) [project_slug=github/demo-cfg8/Azure-demo-APP]

configure8 experts can help you adding data from any source you may have into the file. Just let us know!

Upload tsv File

After adding information in the file its time to bulk onboard your catalog. Go to Settings -> Catalog Import and click the file box under number 2: Upload TSV File:

Select the file and hit Continue. You will be taken to a validation page where we will display the list of Services (rows) and information's to be filled (columns):

If any error is detected we will inform in this screen. If not you can click on Import File and it will push the information into the catalog. A summary of the applied changes will be displayed: