How to migrate data with sharding without foreign key constraints ?

[VadimMitin]

Sorry, if that I repeat question. I have 2 tables in source database(test1 and test2), and test2 has identifier of test1 table. How I can migrate table data from source into another database ? Now I get error from Proxy

2023-10-26 13:16:11 [ERROR] 2023-10-26 10:16:11.278 [j0102p0000c8d59be783b12eca28ce60af30caecbb_Worker-1] freemarker.runtime - Error executing FreeMarker template 2023-10-26 13:16:11 freemarker.core.InvalidReferenceException: The following has evaluated to null or missing: 2023-10-26 13:16:11 ==> confrelid [in template "component/foreign_key/default/get_constraint_cols.ftl" at line 28, column 19] 2023-10-26 13:16:11 2023-10-26 13:16:11 ---- 2023-10-26 13:16:11 Tip: If the failing expression is known to legally refer to something that's sometimes null or missing, either specify a default value like myOptionalVar!myDefault, or use <#if myOptionalVar??>when-present<#else>when-missing</#if>. (These only cover the last step of the expression; to cover the whole expression, use parenthesis: (myOptionalVar.foo)!myDefault, (myOptionalVar.foo)?? 2023-10-26 13:16:11 ---- 2023-10-26 13:16:11 2023-10-26 13:16:11 ---- 2023-10-26 13:16:11 FTL stack trace ("~" means nesting-related): 2023-10-26 13:16:11 - Failed at: ${confrelid} [in template "component/foreign_key/default/get_constraint_cols.ftl" at line 28, column 17] 2023-10-26 13:16:11 ---- 2023-10-26 13:16:11 at freemarker.core.InvalidReferenceException.getInstance(InvalidReferenceException.java:134) 2023-10-26 13:16:11 at freemarker.core.EvalUtil.coerceModelToTextualCommon(EvalUtil.java:481) 2023-10-26 13:16:11 at freemarker.core.EvalUtil.coerceModelToStringOrMarkup(EvalUtil.java:401) 2023-10-26 13:16:11 at freemarker.core.EvalUtil.coerceModelToStringOrMarkup(EvalUtil.java:370) 2023-10-26 13:16:11 at freemarker.core.DollarVariable.calculateInterpolatedStringOrMarkup(DollarVariable.java:100) 2023-10-26 13:16:11 at freemarker.core.DollarVariable.accept(DollarVariable.java:63) 2023-10-26 13:16:11 at freemarker.core.Environment.visit(Environment.java:383) 2023-10-26 13:16:11 at freemarker.core.IteratorBlock$IterationContext.executedNestedContentForCollOrSeqListing(IteratorBlock.java:291) 2023-10-26 13:16:11 at freemarker.core.IteratorBlock$IterationContext.executeNestedContent(IteratorBlock.java:271) 2023-10-26 13:16:11 at freemarker.core.IteratorBlock$IterationContext.accept(IteratorBlock.java:244) 2023-10-26 13:16:11 at freemarker.core.Environment.visitIteratorBlock(Environment.java:657) 2023-10-26 13:16:11 at freemarker.core.IteratorBlock.acceptWithResult(IteratorBlock.java:108) 2023-10-26 13:16:11 at freemarker.core.IteratorBlock.accept(IteratorBlock.java:94) 2023-10-26 13:16:11 at freemarker.core.Environment.visit(Environment.java:347) 2023-10-26 13:16:11 at freemarker.core.Environment.visit(Environment.java:353) 2023-10-26 13:16:11 at freemarker.core.Environment.process(Environment.java:326) 2023-10-26 13:16:11 at freemarker.template.Template.process(Template.java:383) 2023-10-26 13:16:11 at org.apache.shardingsphere.data.pipeline.postgresql.util.PostgreSQLPipelineFreemarkerManager.getSQLByVersion(PostgreSQLPipelineFreemarkerManager.java:77) 2023-10-26 13:16:11 at org.apache.shardingsphere.data.pipeline.postgresql.ddlgenerator.AbstractPostgreSQLDDLAdapter.executeByTemplate(AbstractPostgreSQLDDLAdapter.java:57) 2023-10-26 13:16:11 at org.apache.shardingsphere.data.pipeline.postgresql.ddlgenerator.PostgreSQLConstraintsPropertiesAppender.getForeignKeysCols(PostgreSQLConstraintsPropertiesAppender.java:217) 2023-10-26 13:16:11 at org.apache.shardingsphere.data.pipeline.postgresql.ddlgenerator.PostgreSQLConstraintsPropertiesAppender.getForeignKeys(PostgreSQLConstraintsPropertiesAppender.java:181) 2023-10-26 13:16:11 at org.apache.shardingsphere.data.pipeline.postgresql.ddlgenerator.PostgreSQLConstraintsPropertiesAppender.fetchForeignKeys(PostgreSQLConstraintsPropertiesAppender.java:67) 2023-10-26 13:16:11 at org.apache.shardingsphere.data.pipeline.postgresql.ddlgenerator.PostgreSQLConstraintsPropertiesAppender.append(PostgreSQLConstraintsPropertiesAppender.java:50) 2023-10-26 13:16:11 at org.apache.shardingsphere.data.pipeline.postgresql.ddlgenerator.PostgreSQLCreateTableSQLGenerator.loadMaterials(PostgreSQLCreateTableSQLGenerator.java:54) 2023-10-26 13:16:11 at org.apache.shardingsphere.data.pipeline.postgresql.ddlgenerator.PostgreSQLCreateTableSQLGenerator.generate(PostgreSQLCreateTableSQLGenerator.java:43) 2023-10-26 13:16:11 at org.apache.shardingsphere.data.pipeline.common.metadata.generator.PipelineDDLGenerator.generateLogicDDL(PipelineDDLGenerator.java:79) 2023-10-26 13:16:11 at org.apache.shardingsphere.data.pipeline.core.preparer.datasource.AbstractDataSourcePreparer.getCreateTargetTableSQL(AbstractDataSourcePreparer.java:123) 2023-10-26 13:16:11 at org.apache.shardingsphere.data.pipeline.postgresql.prepare.datasource.PostgreSQLDataSourcePreparer.prepareTargetTables(PostgreSQLDataSourcePreparer.java:40) 2023-10-26 13:16:11 at org.apache.shardingsphere.data.pipeline.core.preparer.PipelineJobPreparerUtils.prepareTargetTables(PipelineJobPreparerUtils.java:123) 2023-10-26 13:16:11 at org.apache.shardingsphere.data.pipeline.scenario.migration.prepare.MigrationJobPreparer.prepareTarget(MigrationJobPreparer.java:168) 2023-10-26 13:16:11 at org.apache.shardingsphere.data.pipeline.scenario.migration.prepare.MigrationJobPreparer.prepareAndCheckTarget(MigrationJobPreparer.java:148) 2023-10-26 13:16:11 at org.apache.shardingsphere.data.pipeline.scenario.migration.prepare.MigrationJobPreparer.prepareAndCheckTargetWithLock(MigrationJobPreparer.java:134) 2023-10-26 13:16:11 at org.apache.shardingsphere.data.pipeline.scenario.migration.prepare.MigrationJobPreparer.prepare(MigrationJobPreparer.java:99) 2023-10-26 13:16:11 at org.apache.shardingsphere.data.pipeline.scenario.migration.MigrationJob.doPrepare(MigrationJob.java:75) 2023-10-26 13:16:11 at org.apache.shardingsphere.data.pipeline.core.job.AbstractPipelineJob.prepare(AbstractPipelineJob.java:91) 2023-10-26 13:16:11 at org.apache.shardingsphere.data.pipeline.core.job.AbstractSimplePipelineJob.execute0(AbstractSimplePipelineJob.java:70) 2023-10-26 13:16:11 at org.apache.shardingsphere.data.pipeline.core.job.AbstractSimplePipelineJob.execute(AbstractSimplePipelineJob.java:50) 2023-10-26 13:16:11 at org.apache.shardingsphere.elasticjob.simple.executor.SimpleJobExecutor.process(SimpleJobExecutor.java:33) 2023-10-26 13:16:11 at org.apache.shardingsphere.elasticjob.simple.executor.SimpleJobExecutor.process(SimpleJobExecutor.java:29) 2023-10-26 13:16:11 at org.apache.shardingsphere.elasticjob.executor.ElasticJobExecutor.process(ElasticJobExecutor.java:173) 2023-10-26 13:16:11 at org.apache.shardingsphere.elasticjob.executor.ElasticJobExecutor.process(ElasticJobExecutor.java:142) 2023-10-26 13:16:11 at org.apache.shardingsphere.elasticjob.executor.ElasticJobExecutor.execute(ElasticJobExecutor.java:124) 2023-10-26 13:16:11 at org.apache.shardingsphere.elasticjob.executor.ElasticJobExecutor.execute(ElasticJobExecutor.java:100) 2023-10-26 13:16:11 at org.apache.shardingsphere.elasticjob.lite.internal.schedule.LiteJob.execute(LiteJob.java:35) 2023-10-26 13:16:11 at org.quartz.core.JobRunShell.run(JobRunShell.java:202) 2023-10-26 13:16:11 at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573) 2023-10-26 13:16:11 [INFO ] 2023-10-26 10:16:11.280 [j0102p0000c8d59be783b12eca28ce60af30caecbb_Worker-1] o.a.s.d.p.s.m.p.MigrationJobPreparer - unlock, jobId=j0102p0000c8d59be783b12eca28ce60af30caecbb, shardingItem=0, cost 1579 ms 2023-10-26 13:16:11 [ERROR] 2023-10-26 10:16:11.282 [j0102p0000c8d59be783b12eca28ce60af30caecbb_Worker-1] o.a.s.e.e.h.g.LogJobErrorHandler - Job 'j0102p0000c8d59be783b12eca28ce60af30caecbb' exception occur in job processing 2023-10-26 13:16:11 freemarker.core.InvalidReferenceException: The following has evaluated to null or missing: 2023-10-26 13:16:11 ==> confrelid [in template "component/foreign_key/default/get_constraint_cols.ftl" at line 28, column 19] 2023-10-26 13:16:11 2023-10-26 13:16:11 ---- 2023-10-26 13:16:11 Tip: If the failing expression is known to legally refer to something that's sometimes null or missing, either specify a default value like myOptionalVar!myDefault, or use <#if myOptionalVar??>when-present<#else>when-missing</#if>. (These only cover the last step of the expression; to cover the whole expression, use parenthesis: (myOptionalVar.foo)!myDefault, (myOptionalVar.foo)?? 2023-10-26 13:16:11 ---- 2023-10-26 13:16:11 2023-10-26 13:16:11 ---- 2023-10-26 13:16:11 FTL stack trace ("~" means nesting-related): 2023-10-26 13:16:11 - Failed at: ${confrelid} [in template "component/foreign_key/default/get_constraint_cols.ftl" at line 28, column 17] 2023-10-26 13:16:11 ---- 2023-10-26 13:16:11 at freemarker.core.InvalidReferenceException.getInstance(InvalidReferenceException.java:134) 2023-10-26 13:16:11 at freemarker.core.EvalUtil.coerceModelToTextualCommon(EvalUtil.java:481) 2023-10-26 13:16:11 at freemarker.core.EvalUtil.coerceModelToStringOrMarkup(EvalUtil.java:401) 2023-10-26 13:16:11 at freemarker.core.EvalUtil.coerceModelToStringOrMarkup(EvalUtil.java:370) 2023-10-26 13:16:11 at freemarker.core.DollarVariable.calculateInterpolatedStringOrMarkup(DollarVariable.java:100) 2023-10-26 13:16:11 at freemarker.core.DollarVariable.accept(DollarVariable.java:63) 2023-10-26 13:16:11 at freemarker.core.Environment.visit(Environment.java:383) 2023-10-26 13:16:11 at freemarker.core.IteratorBlock$IterationContext.executedNestedContentForCollOrSeqListing(IteratorBlock.java:291) 2023-10-26 13:16:11 at freemarker.core.IteratorBlock$IterationContext.executeNestedContent(IteratorBlock.java:271) 2023-10-26 13:16:11 at freemarker.core.IteratorBlock$IterationContext.accept(IteratorBlock.java:244) 2023-10-26 13:16:11 at freemarker.core.Environment.visitIteratorBlock(Environment.java:657) 2023-10-26 13:16:11 at freemarker.core.IteratorBlock.acceptWithResult(IteratorBlock.java:108) 2023-10-26 13:16:11 at freemarker.core.IteratorBlock.accept(IteratorBlock.java:94) 2023-10-26 13:16:11 at freemarker.core.Environment.visit(Environment.java:347) 2023-10-26 13:16:11 at freemarker.core.Environment.visit(Environment.java:353) 2023-10-26 13:16:11 at freemarker.core.Environment.process(Environment.java:326) 2023-10-26 13:16:11 at freemarker.template.Template.process(Template.java:383)