feat(lineage): vscode updated lineage #5495

mykalmax · 2025-10-07T03:03:42Z

No description provided.

benfdking

Quite nervous about these changes because they undo a lot of work that went into making the lineage more up to date and safer with better type safety.

vscode/react/src/pages/help.ts

benfdking · 2025-10-07T08:48:05Z

vscode/react/src/pages/help.ts

+
+export function getNodeTypeColor(nodeType: NodeType) {
+  return {
+    sql: 'bg-lineage-node-type-background-sql',


nit: it might be good for the library to expose these strings so that they are tied together correctly. If you change the library you'll break these downstream without any link.

those colors (for ModelType) are specific to this particular (vscode ext) version of a lineage

vscode/react/src/pages/lineage.tsx

vscode/react/package.json

vscode/react/src/pages/ModelNodeColumn.tsx

benfdking · 2025-10-07T08:58:19Z

vscode/react/src/pages/ModelNodeColumn.tsx

+  const {
+    data: columnLineageData,
+    refetch: getColumnLineage,
+    isFetching: isColumnLineageFetching,
+    error: columnLineageError,
+  } = useApiColumnLineage(nodeId, name, { models_only: true })


These api calls should not leak into the different parts and should be passed down with the correct abstractions.

benfdking · 2025-10-07T08:59:24Z

vscode/react/src/pages/ModelNode.tsx

+import { ModelNodeColumn } from './ModelNodeColumn'
+import type { ModelName as ModelNameType } from '@/domain/models'
+
+export const ModelNode = React.memo(function ModelNode({


Have we tried the react compiler? I think it should do a much better job than us implementing memos by hand?

benfdking

Lots of comments some may be duplicate to what I said before so make reasonable judgements ...

we should add tests

a few other comments on styling

^ can we remove the border around the view and also there's some horizontal space that is being lost to padding.

Can we also not have the lock nodes button, does it do anything?
When you hover over a column, it's not obvious that it is clickable, can we change the cursor on it
If you click a column it is painfully slow to show the column lineage, like multiple seconds slow and sometimes doesn't even show the lineage
Do we need this current tag, I feel like the yellow visualisation does a good enough job of highlighting that, it's the selected node?
If you put the tags on one line, I feel like you would gain quite a bit of real estate

When you select/deselect a column it's pretty unpleasant movement between the two

benfdking · 2025-10-10T08:34:14Z

vscode/bus/src/brand.ts

+/**
+ * Constraint that only accepts branded string types
+ */
+export type BrandedString = string & Brand<string>
+
+/**
+ * BrandedRecord is a type that creates a branded Record type with strict key checking.
+ * This ensures that Record<BrandedKey1, V> is NOT assignable to Record<BrandedKey2, V>
+ *
+ * @example
+ * type ModelFQN = Branded<string, 'ModelFQN'>
+ * type ModelName = Branded<string, 'ModelName'>
+ *
+ * type FQNMap = BrandedRecord<ModelFQN, string>
+ * type NameMap = BrandedRecord<ModelName, string>
+ *
+ * const fqnMap: FQNMap = {}
+ * const nameMap: NameMap = fqnMap // TypeScript error!
+ */
+export type BrandedRecord<K extends BrandedString, V> = Record<K, V> & {
+  readonly __recordKeyBrand: K
+}


nit: For me, I am less convinced about these types. They are an additional thing to learn that actually doesn't stop you from writing any less code. Every time you write one of these, it's another thing for someone new to the codebase to jump to and understand.

BrandedString<"ThingA">
versus
Branded<String, "ThingA">

BrandedRecord<"ThingA", int>
versus
Record<Branded<"ThingA", string>, int>

benfdking · 2025-10-10T08:41:08Z

vscode/react/src/pages/lineage.tsx

  const rpc = useRpc()
  React.useEffect(() => {
-    const fetchFirstTimeModelIfNotSet = async (
+    const fetchFirstTimeModelIfNotSet = async <T extends BrandedString>(


Why the indirection of the generic? In this case, we always want this to return just a model fqn.

benfdking · 2025-10-10T08:42:34Z

vscode/react/src/pages/help.ts

@@ -0,0 +1,36 @@
+import type { ModelType } from '@/api/client'
+
+type NodeType = ModelType | 'cte/subquery'


If you use cte/subquery, then you need to invent a new concept for ModelFQN. A subquery's id is not a model fqn. You need to define a new concept for it.

benfdking · 2025-10-10T08:48:41Z

vscode/react/src/pages/ModelNode.tsx

+  const [showNodeColumns, setShowNodeColumns] = React.useState(showColumns)
+  const [isHovered, setIsHovered] = React.useState(false)
+
+  const nodeId = id as ModelNodeId


nit: To me, this is the first thing you should and we should make a note why we do it.

benfdking · 2025-10-10T08:52:21Z

vscode/react/src/pages/ModelNode.tsx

+    selectedNodeId,
+    selectedNodes,
+    showColumns,
+    fetchingColumns,


I know that we built this dynamically in the past, but I took this away. There are two big problems with this:

The context changes over time; to make sure we don't ever show bad graphs, I reset the graph on every context reload.

The performance of this dynamic fetching makes the lineage unusable if you are using a remote machine, which is true for anyone using Codespaces

benfdking · 2025-10-10T08:55:41Z

vscode/react/src/pages/ModelNode.tsx

+  const {
+    columns,
+    selectedColumns: modelSelectedColumns,
+    columnNames,
+  } = useColumns<ModelFQN, ModelColumnName, Column, ModelColumnID>(
+    selectedColumns,
+    data.name,
+    data.columns,
+  )


Why does this need to be used and again cannot be just gotten from state?

wdym, gotten from state ?

it is calculating for each node

benfdking · 2025-10-10T08:56:59Z

vscode/react/src/pages/ModelNode.tsx

+  )
+
+  const hasSelectedColumns = selectedColumns.intersection(columnNames).size > 0
+  const hasFetchingColumns = fetchingColumns.intersection(columnNames).size > 0


Similar to my comments above, I don't think we should have dynamic data graphs.

The graph should either be loading fully or not.

it is fetching data for CLL, we cant make send it upfront, yet

benfdking · 2025-10-10T08:59:47Z

vscode/react/src/pages/ModelNode.tsx

+    >
+      <NodeAppendix
+        position="top"
+        className="bg-lineage-node-appendix-background"


I have really no problem with these but if they are coming from the common it might be worth just making them constants.

benfdking · 2025-10-10T09:01:01Z

vscode/react/src/pages/ModelNode.tsx

+          className="overflow-visible"
+          onMouseEnter={() => setIsHovered(true)}
+          onMouseLeave={() => setIsHovered(false)}


I am always going to fight to not have visual stuff be JS dependent because it's slow 😛 but I don't have a good view of whether it is possible. Might be a question for Claude 😛

benfdking · 2025-10-10T09:08:14Z

vscode/react/src/pages/ModelNodeColumn.tsx

+      type={type}
+      description={description}
+      className={cn(
+        'ModelNodeColumn',


don't we use kebab casing for css ModelNodeColumn?

we do, but for component class i prefer component name as css class to find it faster if needed

benfdking requested changes Oct 7, 2025

View reviewed changes

mykalmax force-pushed the mykalmax/web_common-vscode-lineage branch 2 times, most recently from 57e35b0 to 21f636c Compare October 10, 2025 00:26

benfdking reviewed Oct 10, 2025

View reviewed changes

mykalmax force-pushed the mykalmax/web_common-vscode-lineage branch 2 times, most recently from deb8585 to 40f65ad Compare October 13, 2025 15:07

mykalmax added 19 commits October 13, 2025 15:56

set up lineage for vscode ext

df94ad4

update packages

587aad1

remove unused

750e5fe

update css

2df52bd

simplify colors

ee3067e

clean up

f8b802c

adjust colors

d52ea72

remove grpah component from vscode ext

a51319c

fix node selection

e513e10

some pr comments

a062adc

package

dccfd6d

include common package directly

8cc3f6f

pr comments

fde4ae6

types

81d2ba0

formatting

a6bd413

formatting

1fbe274

clean up

52a52b5

fix test

8ebd24d

formattign

59c4c2f

mykalmax force-pushed the mykalmax/web_common-vscode-lineage branch from 5d345c5 to 59c4c2f Compare October 13, 2025 22:58

		@@ -0,0 +1,36 @@
		import type { ModelType } from '@/api/client'

		type NodeType = ModelType \| 'cte/subquery'

feat(lineage): vscode updated lineage #5495

Are you sure you want to change the base?

feat(lineage): vscode updated lineage #5495

Conversation

mykalmax commented Oct 7, 2025

Uh oh!

benfdking left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

benfdking left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

benfdking left a comment •

edited

Loading