Mynd for data engineers
Data Engineer, as it will appear in the app, in the next version.
A data engineer builds the paths data travels: sources, pipelines, storage and quality. When something breaks, it is necessary to know what changed and when.
The ready-made structure
Score 89 out of 100. What the score means.
The structure starts with Data, the centre of the work, divided into Sources, Collection and Quality. Then come Pipelines, Storage, Integration, Operations, Governance and AI. Each record keeps its full path, for example Data, Sources, Internal, and can be found by any of the three levels.
The full tree is at the end of this page, with the names as they are in the app.
Suggested Context phrases
The structure says what each record is about. The Context phrases, which each person creates, say the rest. For this profession, some that tend to serve:
Pipeline failure Source change Quality Resolved
A report that sends itself
A search gathers the week's failures and source changes. Every Friday, the teams that use the data receive the summary.
What you gain: Data users know what changed before they find odd numbers.
The card with the map of data sources goes to each new analyst.
In this version, Start, each person has 14 automatic search sends and 14 card sends; in Run, they will be able to have as many as they need. Each send has its own search, its own recipients and its own frequency: daily, weekly, monthly or on a date. Emails go out from the person's own email account, and each recipient receives their own.
The full tree
Data3 Types
- Data/SourcesInternalExternalStreaming
- Data/CollectionIngestionExtractionIntegration
- Data/QualityValidationConsistencyMonitoring
Pipelines3 Types
- Pipelines/DesignArchitectureWorkflowsScheduling
- Pipelines/ProcessingBatchRealtimeTransformation
- Pipelines/OptimizationPerformanceScalabilityReliability
Storage3 Types
- Storage/DatabasesRelationalNoSQLGraph
- Storage/WarehousesModelingPartitioningAccess
- Storage/LakesRawCuratedGoverned
Integration3 Types
- Integration/SystemsApplicationsPlatformsServices
- Integration/InterfacesAPIsEventsMessaging
- Integration/SynchronizationReplicationConsistencyRecovery
Operations3 Types
- Operations/InfrastructureComputeStorageNetwork
- Operations/MonitoringMetricsLogsAlerts
- Operations/MaintenanceUpdatesAutomationOptimization
Governance3 Types
- Governance/SecurityAccessEncryptionProtection
- Governance/CompliancePoliciesAuditsRetention
- Governance/PrivacyConsentMaskingControl
AI3 Types
- AI/PipelinesEmbeddingsChunkingIndexing
- AI/QualityLabelsDeduplicationDrift
- AI/AssistantsSQLTransformationsDocs
