Skip to content

Use Google's project MediaPipe Face Mesh as FACS tracker #33

@NumesSanguis

Description

@NumesSanguis

Overview

As @fire shared in issue 27, Google has released MediaPipe Face Mesh as a Python library (code).

As this solution seems to be fast and is Apache-2.0 Licensed, it's interesting to consider this as a replacement for OpenFace. The tracking output is 468 3D coordinates (x, y, z) for a single mask (1 image or 1 frame in a video), which looks like:
mediapipe_facemesh_output

To be useful for this project, the changes in these masks need to be mapped to Action Unit (AU) values according to the Facial Action Coding System (FACS) model.
More info - Demonstration of facial movement per AU.

This can be tricky, because the deformation in the tracking mesh when 2 AUs are active at the same time, won't be always just be adding the deformation of the AUs individually (no linear relation between e.g. AU12+AU14 compared to AU12 and AU14 separately). In practice a linear approach might be good enough though.

A paper by OpenFace on how they did Facial Action Unit detection:

MediaPipe Face Mesh output

A single face mesh tracking a single person in 1 image/frame has 468*3 values. A single landmark:

x: 0.4966168701648712
y: 0.694128692150116
z: -0.11288105696439743

Note that this output is normalized between [-1, 1].

Full landmark output single mesh (CLICK ME)

landmark {
  x: 0.4966168701648712
  y: 0.694128692150116
  z: -0.11288105696439743
}
landmark {
  x: 0.4816158711910248
  y: 0.6072000861167908
  z: -0.21528172492980957
}
landmark {
  x: 0.48926183581352234
  y: 0.6384062170982361
  z: -0.1202792227268219
}
landmark {
  x: 0.4422526955604553
  y: 0.5044909715652466
  z: -0.15802232921123505
}
landmark {
  x: 0.47746509313583374
  y: 0.5734008550643921
  z: -0.22716723382472992
}
landmark {
  x: 0.47410061955451965
  y: 0.5310384035110474
  z: -0.2105686366558075
}
landmark {
  x: 0.46738678216934204
  y: 0.4304444193840027
  z: -0.10389776527881622
}
landmark {
  x: 0.21002808213233948
  y: 0.4479711651802063
  z: 0.06500452011823654
}
landmark {
  x: 0.45994189381599426
  y: 0.353225439786911
  z: -0.07487861067056656
}
landmark {
  x: 0.45476841926574707
  y: 0.3084910809993744
  z: -0.08058346062898636
}
landmark {
  x: 0.44055771827697754
  y: 0.14465117454528809
  z: -0.038242045789957047
}
landmark {
  x: 0.49868014454841614
  y: 0.7090015411376953
  z: -0.10894951224327087
}
landmark {
  x: 0.5007204413414001
  y: 0.719012439250946
  z: -0.09897053241729736
}
landmark {
  x: 0.5020402073860168
  y: 0.7213298082351685
  z: -0.08386010676622391
}
landmark {
  x: 0.5109937787055969
  y: 0.7895573973655701
  z: -0.05537683516740799
}
landmark {
  x: 0.5119189023971558
  y: 0.8027503490447998
  z: -0.06261561065912247
}
landmark {
  x: 0.5133177638053894
  y: 0.8185326457023621
  z: -0.0709986463189125
}
landmark {
  x: 0.5152280330657959
  y: 0.8369268774986267
  z: -0.06422298401594162
}
landmark {
  x: 0.5181258916854858
  y: 0.8634383678436279
  z: -0.03230631723999977
}
landmark {
  x: 0.48471203446388245
  y: 0.6251006722450256
  z: -0.19659900665283203
}
landmark {
  x: 0.4531421959400177
  y: 0.6249157190322876
  z: -0.143781840801239
}
landmark {
  x: 0.06447488069534302
  y: 0.339648574590683
  z: 0.25507593154907227
}
landmark {
  x: 0.31962549686431885
  y: 0.46791741251945496
  z: 0.02018388733267784
}
landmark {
  x: 0.28245049715042114
  y: 0.47231966257095337
  z: 0.025203807279467583
}
landmark {
  x: 0.24612677097320557
  y: 0.47224801778793335
  z: 0.0373702310025692
}
landmark {
  x: 0.19500768184661865
  y: 0.45980608463287354
  z: 0.07198791205883026
}
landmark {
  x: 0.349759966135025
  y: 0.45816195011138916
  z: 0.022274039685726166
}
landmark {
  x: 0.26011133193969727
  y: 0.389184832572937
  z: 0.010085818357765675
}
landmark {
  x: 0.2995329797267914
  y: 0.389138400554657
  z: 0.008145865984261036
}
landmark {
  x: 0.22314757108688354
  y: 0.39661189913749695
  z: 0.02493373677134514
}
landmark {
  x: 0.19949522614479065
  y: 0.40996548533439636
  z: 0.0419645681977272
}
landmark {
  x: 0.16591835021972656
  y: 0.48124581575393677
  z: 0.09229511767625809
}
landmark {
  x: 0.3812064528465271
  y: 0.9071664214134216
  z: 0.021440977230668068
}
landmark {
  x: 0.19616428017616272
  y: 0.4427569508552551
  z: 0.07880867272615433
}
landmark {
  x: 0.062373995780944824
  y: 0.48197007179260254
  z: 0.26922407746315
}
landmark {
  x: 0.12454494833946228
  y: 0.465819776058197
  z: 0.13364587724208832
}
landmark {
  x: 0.2905968725681305
  y: 0.5917161703109741
  z: -0.022236289456486702
}
landmark {
  x: 0.4381605386734009
  y: 0.6930720210075378
  z: -0.10508478432893753
}
landmark {
  x: 0.4468565285205841
  y: 0.7214252352714539
  z: -0.08921617269515991
}
landmark {
  x: 0.38093671202659607
  y: 0.7045804262161255
  z: -0.07571420818567276
}
landmark {
  x: 0.3459462523460388
  y: 0.717242956161499
  z: -0.033613234758377075
}
landmark {
  x: 0.4050546884536743
  y: 0.7251520156860352
  z: -0.06517799943685532
}
landmark {
  x: 0.37378859519958496
  y: 0.7300823926925659
  z: -0.02689925581216812
}
landmark {
  x: 0.3168252110481262
  y: 0.7854228019714355
  z: 0.044089797884225845
}
landmark {
  x: 0.45147567987442017
  y: 0.6086890697479248
  z: -0.21051371097564697
}
landmark {
  x: 0.4435341954231262
  y: 0.5764272809028625
  z: -0.22110529243946075
}
landmark {
  x: 0.13545703887939453
  y: 0.3788211941719055
  z: 0.056899067014455795
}
landmark {
  x: 0.3665546476840973
  y: 0.5107970833778381
  z: -0.027126528322696686
}
landmark {
  x: 0.36413857340812683
  y: 0.6033427715301514
  z: -0.11284682154655457
}
landmark {
  x: 0.36061158776283264
  y: 0.5862759947776794
  z: -0.10040289163589478
}
landmark {
  x: 0.1746581494808197
  y: 0.5945844054222107
  z: 0.03535617142915726
}
landmark {
  x: 0.4421435296535492
  y: 0.5369409322738647
  z: -0.1966618001461029
}
landmark {
  x: 0.2176908552646637
  y: 0.33583539724349976
  z: -0.014324278570711613
}
landmark {
  x: 0.16782251000404358
  y: 0.35119521617889404
  z: 0.01882544904947281
}
landmark {
  x: 0.09129291772842407
  y: 0.2760707437992096
  z: 0.179345965385437
}
landmark {
  x: 0.3814221918582916
  y: 0.3533117473125458
  z: -0.05903608724474907
}
landmark {
  x: 0.3350331783294678
  y: 0.3979620933532715
  z: 0.016856539994478226
}
landmark {
  x: 0.2812685966491699
  y: 0.745897650718689
  z: 0.04884406924247742
}
landmark {
  x: 0.1452232003211975
  y: 0.7861756086349487
  z: 0.3580620586872101
}
landmark {
  x: 0.394314706325531
  y: 0.619657576084137
  z: -0.10019056499004364
}
landmark {
  x: 0.4270922541618347
  y: 0.6279042959213257
  z: -0.10671922564506531
}
landmark {
  x: 0.307145357131958
  y: 0.7430539727210999
  z: 0.049138471484184265
}
landmark {
  x: 0.32793134450912476
  y: 0.7431167960166931
  z: 0.03924649953842163
}
landmark {
  x: 0.14394527673721313
  y: 0.3307373821735382
  z: 0.03859829157590866
}
landmark {
  x: 0.3629467487335205
  y: 0.6155763268470764
  z: -0.09240224212408066
}
landmark {
  x: 0.28702664375305176
  y: 0.3346884250640869
  z: -0.04128420352935791
}
landmark {
  x: 0.27642199397087097
  y: 0.30713507533073425
  z: -0.04866618663072586
}
landmark {
  x: 0.22702038288116455
  y: 0.1813761293888092
  z: 0.023917950689792633
}
landmark {
  x: 0.11776086688041687
  y: 0.3024034798145294
  z: 0.10050833225250244
}
landmark {
  x: 0.2545614540576935
  y: 0.24051368236541748
  z: -0.01349948812276125
}
landmark {
  x: 0.11170089244842529
  y: 0.36651477217674255
  z: 0.09082090854644775
}
landmark {
  x: 0.08806124329566956
  y: 0.35485026240348816
  z: 0.16924317181110382
}
landmark {
  x: 0.44142016768455505
  y: 0.71026611328125
  z: -0.10029156506061554
}
landmark {
  x: 0.39390215277671814
  y: 0.7174962162971497
  z: -0.07107415795326233
}
landmark {
  x: 0.35953131318092346
  y: 0.7237300276756287
  z: -0.032877374440431595
}
landmark {
  x: 0.40349727869033813
  y: 0.6242803335189819
  z: -0.09295294433832169
}
landmark {
  x: 0.3165252208709717
  y: 0.7428818941116333
  z: 0.043773844838142395
}
landmark {
  x: 0.3406851887702942
  y: 0.7669117450714111
  z: 0.023747634142637253
}
landmark {
  x: 0.33578944206237793
  y: 0.7433992028236389
  z: 0.03797805681824684
}
landmark {
  x: 0.4140060544013977
  y: 0.6110385656356812
  z: -0.15295974910259247
}
landmark {
  x: 0.38158589601516724
  y: 0.7267777323722839
  z: -0.022487040609121323
}
landmark {
  x: 0.41331616044044495
  y: 0.724240779876709
  z: -0.0519661121070385
}
landmark {
  x: 0.4527042806148529
  y: 0.7233290076255798
  z: -0.07439269125461578
}
landmark {
  x: 0.45655256509780884
  y: 0.8648825287818909
  z: -0.02545144222676754
}
landmark {
  x: 0.4548318684101105
  y: 0.8358229398727417
  z: -0.05548474192619324
}
landmark {
  x: 0.4551684558391571
  y: 0.8169925212860107
  z: -0.059868987649679184
}
landmark {
  x: 0.457076758146286
  y: 0.8008837699890137
  z: -0.05086479336023331
}
landmark {
  x: 0.4605860710144043
  y: 0.7895804643630981
  z: -0.0449383407831192
}
landmark {
  x: 0.38548389077186584
  y: 0.7749221324920654
  z: 9.880734432954341e-05
}
landmark {
  x: 0.3780013620853424
  y: 0.7787157893180847
  z: -0.005501376464962959
}
landmark {
  x: 0.36844146251678467
  y: 0.7868350148200989
  z: -0.009895708411931992
}
landmark {
  x: 0.3618044853210449
  y: 0.7972811460494995
  z: -0.0017165766330435872
}
landmark {
  x: 0.3194640278816223
  y: 0.6933881044387817
  z: -0.021864671260118484
}
landmark {
  x: 0.08811995387077332
  y: 0.6305686235427856
  z: 0.4192250669002533
}
landmark {
  x: 0.48704424500465393
  y: 0.6317288279533386
  z: -0.14750073850154877
}
landmark {
  x: 0.36362749338150024
  y: 0.76473468542099
  z: 0.025699688121676445
}
landmark {
  x: 0.35120195150375366
  y: 0.7645043134689331
  z: 0.02254905179142952
}
landmark {
  x: 0.4399850070476532
  y: 0.6405894756317139
  z: -0.10764143615961075
}
landmark {
  x: 0.375720739364624
  y: 0.6322762966156006
  z: -0.06489422917366028
}
landmark {
  x: 0.4340003728866577
  y: 0.634506344795227
  z: -0.10850734263658524
}
landmark {
  x: 0.32653310894966125
  y: 0.5300822854042053
  z: -0.012731536291539669
}
landmark {
  x: 0.2632318139076233
  y: 0.5526027083396912
  z: 0.00027940748259425163
}
landmark {
  x: 0.3542672097682953
  y: 0.5996798276901245
  z: -0.07833056151866913
}
landmark {
  x: 0.14197686314582825
  y: 0.2222341001033783
  z: 0.09827900677919388
}
landmark {
  x: 0.16972631216049194
  y: 0.26265406608581543
  z: 0.04034826159477234
}
landmark {
  x: 0.20060905814170837
  y: 0.3098430633544922
  z: -0.006581175606697798
}
landmark {
  x: 0.35248780250549316
  y: 0.8181872367858887
  z: 0.022957870736718178
}
landmark {
  x: 0.36348414421081543
  y: 0.3094140589237213
  z: -0.07173669338226318
}
landmark {
  x: 0.3437490463256836
  y: 0.23109906911849976
  z: -0.047013651579618454
}
landmark {
  x: 0.3235037922859192
  y: 0.15789145231246948
  z: -0.02233033999800682
}
landmark {
  x: 0.21419644355773926
  y: 0.4688110947608948
  z: 0.05552274361252785
}
landmark {
  x: 0.12787866592407227
  y: 0.5059182047843933
  z: 0.1152096688747406
}
landmark {
  x: 0.36812901496887207
  y: 0.44856852293014526
  z: 0.025466639548540115
}
landmark {
  x: 0.15955564379692078
  y: 0.42317861318588257
  z: 0.07820062339305878
}
landmark {
  x: 0.3913554549217224
  y: 0.4891124963760376
  z: -0.04222859442234039
}
landmark {
  x: 0.38729652762413025
  y: 0.5918189883232117
  z: -0.15105858445167542
}
landmark {
  x: 0.09255272150039673
  y: 0.5330958962440491
  z: 0.16015303134918213
}
landmark {
  x: 0.1535634994506836
  y: 0.5218596458435059
  z: 0.07908810675144196
}
landmark {
  x: 0.20109590888023376
  y: 0.5274339914321899
  z: 0.04150325804948807
}
landmark {
  x: 0.27242377400398254
  y: 0.5198872685432434
  z: 0.020176483318209648
}
landmark {
  x: 0.326151967048645
  y: 0.5059536695480347
  z: 0.008998933248221874
}
landmark {
  x: 0.36502182483673096
  y: 0.48930275440216064
  z: -0.0038086730055510998
}
landmark {
  x: 0.4342425763607025
  y: 0.44238021969795227
  z: -0.08627469837665558
}
landmark {
  x: 0.09740403294563293
  y: 0.5966350436210632
  z: 0.1479249894618988
}
landmark {
  x: 0.12877383828163147
  y: 0.42056652903556824
  z: 0.09765920042991638
}
landmark {
  x: 0.46656739711761475
  y: 0.6248356699943542
  z: -0.19346565008163452
}
landmark {
  x: 0.3698480725288391
  y: 0.5399408936500549
  z: -0.046744342893362045
}
landmark {
  x: 0.0581088662147522
  y: 0.4841393232345581
  z: 0.3943922519683838
}
landmark {
  x: 0.3915879726409912
  y: 0.47015851736068726
  z: -0.011566780507564545
}
landmark {
  x: 0.3505880832672119
  y: 0.6014127135276794
  z: -0.03828176483511925
}
landmark {
  x: 0.18235325813293457
  y: 0.44442564249038696
  z: 0.08718197047710419
}
landmark {
  x: 0.38435113430023193
  y: 0.5708659887313843
  z: -0.13555653393268585
}
landmark {
  x: 0.1089932918548584
  y: 0.7077124118804932
  z: 0.39826714992523193
}
landmark {
  x: 0.3669191300868988
  y: 0.4378426671028137
  z: 0.03585735708475113
}
landmark {
  x: 0.41278910636901855
  y: 0.5510349273681641
  z: -0.17305362224578857
}
landmark {
  x: 0.22341036796569824
  y: 0.8427634239196777
  z: 0.1659410148859024
}
landmark {
  x: 0.24016138911247253
  y: 0.8845473527908325
  z: 0.22497878968715668
}
landmark {
  x: 0.07366946339607239
  y: 0.6184796690940857
  z: 0.28494521975517273
}
landmark {
  x: 0.16882428526878357
  y: 0.8082045912742615
  z: 0.208804190158844
}
landmark {
  x: 0.07401168346405029
  y: 0.41425713896751404
  z: 0.23039989173412323
}
landmark {
  x: 0.38395026326179504
  y: 0.9384512305259705
  z: 0.038852691650390625
}
landmark {
  x: 0.4726627469062805
  y: 0.6314657926559448
  z: -0.14449647068977356
}
landmark {
  x: 0.3361490070819855
  y: 0.5637073516845703
  z: -0.03239408880472183
}
landmark {
  x: 0.09436336159706116
  y: 0.47562286257743835
  z: 0.1737174242734909
}
landmark {
  x: 0.24819177389144897
  y: 0.4541197419166565
  z: 0.03995387256145477
}
landmark {
  x: 0.28098762035369873
  y: 0.45491164922714233
  z: 0.02891157753765583
}
landmark {
  x: 0.33149421215057373
  y: 0.7714042663574219
  z: 0.0297850389033556
}
landmark {
  x: 0.1068907082080841
  y: 0.6629747748374939
  z: 0.15780605375766754
}
landmark {
  x: 0.4563775956630707
  y: 0.9773697257041931
  z: 0.038378626108169556
}
landmark {
  x: 0.34909868240356445
  y: 0.9424223899841309
  z: 0.12394849210977554
}
landmark {
  x: 0.30104225873947144
  y: 0.9184457063674927
  z: 0.1700422614812851
}
landmark {
  x: 0.4470309317111969
  y: 0.22389227151870728
  z: -0.061588384211063385
}
landmark {
  x: 0.5305874347686768
  y: 0.9793791770935059
  z: 0.02257702499628067
}
landmark {
  x: 0.3115778863430023
  y: 0.4516664147377014
  z: 0.02560044266283512
}
landmark {
  x: 0.34036505222320557
  y: 0.44540536403656006
  z: 0.029373547062277794
}
landmark {
  x: 0.35830408334732056
  y: 0.4412270188331604
  z: 0.03582789748907089
}
landmark {
  x: 0.10178148746490479
  y: 0.41643524169921875
  z: 0.13728414475917816
}
landmark {
  x: 0.3333684802055359
  y: 0.4224480390548706
  z: 0.023977747187018394
}
landmark {
  x: 0.30116382241249084
  y: 0.4180963933467865
  z: 0.019164299592375755
}
landmark {
  x: 0.2705670893192291
  y: 0.41848498582839966
  z: 0.022922012954950333
}
landmark {
  x: 0.23962080478668213
  y: 0.42437681555747986
  z: 0.03419533744454384
}
landmark {
  x: 0.21877902746200562
  y: 0.4323749542236328
  z: 0.047679878771305084
}
landmark {
  x: 0.0542866587638855
  y: 0.40489107370376587
  z: 0.33343178033828735
}
landmark {
  x: 0.2258961796760559
  y: 0.45137596130371094
  z: 0.0531965009868145
}
landmark {
  x: 0.492154061794281
  y: 0.6571369767189026
  z: -0.10802311450242996
}
landmark {
  x: 0.35693883895874023
  y: 0.6745429039001465
  z: -0.05522865429520607
}
landmark {
  x: 0.39345255494117737
  y: 0.6159301996231079
  z: -0.11440737545490265
}
landmark {
  x: 0.43464553356170654
  y: 0.6622862219810486
  z: -0.10015508532524109
}
landmark {
  x: 0.4644954800605774
  y: 0.3917056918144226
  z: -0.07517721503973007
}
landmark {
  x: 0.28190237283706665
  y: 0.879287838935852
  z: 0.1305902898311615
}
landmark {
  x: 0.33249735832214355
  y: 0.9081810712814331
  z: 0.08741878718137741
}
landmark {
  x: 0.44906002283096313
  y: 0.9580975770950317
  z: -0.003724485170096159
}
landmark {
  x: 0.1908087432384491
  y: 0.8453988432884216
  z: 0.29279109835624695
}
landmark {
  x: 0.35627108812332153
  y: 0.431122362613678
  z: 0.03127270191907883
}
landmark {
  x: 0.41472727060317993
  y: 0.49441081285476685
  z: -0.09768738597631454
}
landmark {
  x: 0.5268312692642212
  y: 0.9583680629730225
  z: -0.01725558005273342
}
landmark {
  x: 0.3982726037502289
  y: 0.9635181427001953
  z: 0.07418964058160782
}
landmark {
  x: 0.09216967225074768
  y: 0.6903148293495178
  z: 0.27550944685935974
}
landmark {
  x: 0.41915491223335266
  y: 0.7839378118515015
  z: -0.02422095090150833
}
landmark {
  x: 0.4124149680137634
  y: 0.7920427918434143
  z: -0.02981782704591751
}
landmark {
  x: 0.4065060317516327
  y: 0.8049683570861816
  z: -0.03447248786687851
}
landmark {
  x: 0.4030742049217224
  y: 0.8215083479881287
  z: -0.02789442613720894
}
landmark {
  x: 0.39689958095550537
  y: 0.8466787338256836
  z: -0.003043192205950618
}
landmark {
  x: 0.3447519540786743
  y: 0.7349370121955872
  z: 0.008834567852318287
}
landmark {
  x: 0.3311789631843567
  y: 0.7321010828018188
  z: 0.010857325978577137
}
landmark {
  x: 0.3195076584815979
  y: 0.7279363870620728
  z: 0.0107242651283741
}
landmark {
  x: 0.2904130518436432
  y: 0.7160662412643433
  z: 0.01643066480755806
}
landmark {
  x: 0.15070724487304688
  y: 0.6711384654045105
  z: 0.0781637653708458
}
landmark {
  x: 0.4137824773788452
  y: 0.4639121890068054
  z: -0.06148932874202728
}
landmark {
  x: 0.3897281885147095
  y: 0.4090869128704071
  z: 0.00622992729768157
}
landmark {
  x: 0.36975330114364624
  y: 0.41790977120399475
  z: 0.02349216863512993
}
landmark {
  x: 0.35541102290153503
  y: 0.7306694984436035
  z: 0.010990123264491558
}
landmark {
  x: 0.164760559797287
  y: 0.7537547945976257
  z: 0.13850508630275726
}
landmark {
  x: 0.41884663701057434
  y: 0.40231144428253174
  z: -0.046456996351480484
}
landmark {
  x: 0.38834986090660095
  y: 0.8745599389076233
  z: 0.010446273721754551
}
landmark {
  x: 0.4720120429992676
  y: 0.4969934821128845
  z: -0.17284521460533142
}
landmark {
  x: 0.43800702691078186
  y: 0.4754502773284912
  z: -0.125311017036438
}
landmark {
  x: 0.46979230642318726
  y: 0.4655134677886963
  z: -0.1363074779510498
}
landmark {
  x: 0.3925747275352478
  y: 0.544338583946228
  z: -0.09969377517700195
}
landmark {
  x: 0.5232637524604797
  y: 0.9283697009086609
  z: -0.03395295888185501
}
landmark {
  x: 0.5202719569206238
  y: 0.8925750255584717
  z: -0.03351389244198799
}
landmark {
  x: 0.45128118991851807
  y: 0.8941555619239807
  z: -0.02066991664469242
}
landmark {
  x: 0.2987120747566223
  y: 0.8035737872123718
  z: 0.06246912106871605
}
landmark {
  x: 0.32278764247894287
  y: 0.628929078578949
  z: -0.02323312498629093
}
landmark {
  x: 0.3410077691078186
  y: 0.8435471057891846
  z: 0.04044443741440773
}
landmark {
  x: 0.2297494113445282
  y: 0.6335470676422119
  z: -0.0015491320518776774
}
landmark {
  x: 0.28704535961151123
  y: 0.6627032160758972
  z: -0.005966142285615206
}
landmark {
  x: 0.2043558955192566
  y: 0.6869828104972839
  z: 0.03605557605624199
}
landmark {
  x: 0.44654372334480286
  y: 0.9296964406967163
  z: -0.01958140730857849
}
landmark {
  x: 0.3718560039997101
  y: 0.5627002120018005
  z: -0.07194902747869492
}
landmark {
  x: 0.2805273234844208
  y: 0.8365193605422974
  z: 0.09268306195735931
}
landmark {
  x: 0.3318604826927185
  y: 0.8726451396942139
  z: 0.06202562153339386
}
landmark {
  x: 0.25948265194892883
  y: 0.760127067565918
  z: 0.06235231086611748
}
landmark {
  x: 0.12568554282188416
  y: 0.7197843790054321
  z: 0.17079582810401917
}
landmark {
  x: 0.21877965331077576
  y: 0.7787196636199951
  z: 0.09545453637838364
}
landmark {
  x: 0.12219688296318054
  y: 0.7549031376838684
  z: 0.25807473063468933
}
landmark {
  x: 0.25807932019233704
  y: 0.7004751563072205
  z: 0.02069525420665741
}
landmark {
  x: 0.3937021791934967
  y: 0.5172567367553711
  z: -0.07004125416278839
}
landmark {
  x: 0.40121617913246155
  y: 0.6062255501747131
  z: -0.1600225418806076
}
landmark {
  x: 0.3769870102405548
  y: 0.6137251853942871
  z: -0.11848873645067215
}
landmark {
  x: 0.4142044186592102
  y: 0.5831300020217896
  z: -0.1871527135372162
}
landmark {
  x: 0.3571854829788208
  y: 0.3837273418903351
  z: -0.0017102970741689205
}
landmark {
  x: 0.2956382930278778
  y: 0.3691142201423645
  z: -0.003822853323072195
}
landmark {
  x: 0.2431805431842804
  y: 0.3689405918121338
  z: 0.0017890947638079524
}
landmark {
  x: 0.20002737641334534
  y: 0.3778613805770874
  z: 0.0207314882427454
}
landmark {
  x: 0.17119255661964417
  y: 0.39538031816482544
  z: 0.046379540115594864
}
landmark {
  x: 0.15769413113594055
  y: 0.4544641971588135
  z: 0.10427386313676834
}
landmark {
  x: 0.06413432955741882
  y: 0.5501613616943359
  z: 0.28021129965782166
}
landmark {
  x: 0.18775427341461182
  y: 0.49140042066574097
  z: 0.07144352793693542
}
landmark {
  x: 0.22720354795455933
  y: 0.49667030572891235
  z: 0.04704747349023819
}
landmark {
  x: 0.27706509828567505
  y: 0.49465706944465637
  z: 0.029038190841674805
}
landmark {
  x: 0.32410991191864014
  y: 0.48578399419784546
  z: 0.018875757232308388
}
landmark {
  x: 0.3597770035266876
  y: 0.4729805588722229
  z: 0.014622870832681656
}
landmark {
  x: 0.3826742172241211
  y: 0.4602789282798767
  z: 0.01001096609979868
}
landmark {
  x: 0.07308945059776306
  y: 0.5572210550308228
  z: 0.4202417731285095
}
landmark {
  x: 0.3781541585922241
  y: 0.6203401684761047
  z: -0.09991267323493958
}
landmark {
  x: 0.41601312160491943
  y: 0.5217124223709106
  z: -0.13004370033740997
}
landmark {
  x: 0.42697784304618835
  y: 0.60621577501297
  z: -0.19082331657409668
}
landmark {
  x: 0.44973915815353394
  y: 0.6207560896873474
  z: -0.17728112637996674
}
landmark {
  x: 0.42836445569992065
  y: 0.6117011904716492
  z: -0.17246049642562866
}
landmark {
  x: 0.3879000246524811
  y: 0.6280851364135742
  z: -0.08750980347394943
}
landmark {
  x: 0.45556584000587463
  y: 0.6235464215278625
  z: -0.18769052624702454
}
landmark {
  x: 0.4613252580165863
  y: 0.6296263337135315
  z: -0.14377787709236145
}
landmark {
  x: 0.3795766532421112
  y: 0.43754279613494873
  z: 0.0279754139482975
}
landmark {
  x: 0.39896324276924133
  y: 0.44095879793167114
  z: 0.003275311319157481
}
landmark {
  x: 0.4094507694244385
  y: 0.44369012117385864
  z: -0.02484704740345478
}
landmark {
  x: 0.2062455117702484
  y: 0.43818193674087524
  z: 0.06118197739124298
}
landmark {
  x: 0.18186673521995544
  y: 0.4249523878097534
  z: 0.06467242538928986
}
landmark {
  x: 0.5035853981971741
  y: 0.4999598562717438
  z: -0.16247594356536865
}
landmark {
  x: 0.7413772344589233
  y: 0.40681785345077515
  z: 0.03183317929506302
}
landmark {
  x: 0.5206395387649536
  y: 0.6203845143318176
  z: -0.14702844619750977
}
landmark {
  x: 0.8921987414360046
  y: 0.27469971776008606
  z: 0.20559513568878174
}
landmark {
  x: 0.6345242261886597
  y: 0.443726509809494
  z: -0.0012637971667572856
}
landmark {
  x: 0.6732664704322815
  y: 0.4422304034233093
  z: -0.0007405339856632054
}
landmark {
  x: 0.7107070684432983
  y: 0.4366280734539032
  z: 0.007454411592334509
}
landmark {
  x: 0.7611681222915649
  y: 0.41643911600112915
  z: 0.03679535910487175
}
landmark {
  x: 0.6014629602432251
  y: 0.4388699531555176
  z: 0.005170207004994154
}
landmark {
  x: 0.6754707098007202
  y: 0.35596051812171936
  z: -0.016539370641112328
}
landmark {
  x: 0.6357946395874023
  y: 0.362026184797287
  z: -0.012676878832280636
}
landmark {
  x: 0.7149510979652405
  y: 0.35779958963394165
  z: -0.005732965189963579
}
landmark {
  x: 0.7429794669151306
  y: 0.3676159083843231
  z: 0.00850668828934431
}
landmark {
  x: 0.798031747341156
  y: 0.4332960247993469
  z: 0.054415538907051086
}
landmark {
  x: 0.6666548848152161
  y: 0.8896511793136597
  z: 0.006440763361752033
}
landmark {
  x: 0.7551484107971191
  y: 0.39951521158218384
  z: 0.044434018433094025
}
landmark {
  x: 0.9230958223342896
  y: 0.41696596145629883
  z: 0.21836701035499573
}
landmark {
  x: 0.8410305976867676
  y: 0.41109052300453186
  z: 0.09126061201095581
}
landmark {
  x: 0.6870469450950623
  y: 0.5627361536026001
  z: -0.04433140903711319
}
landmark {
  x: 0.5558522939682007
  y: 0.6847144365310669
  z: -0.11138556152582169
}
landmark {
  x: 0.5563403964042664
  y: 0.7140500545501709
  z: -0.0948861837387085
}
landmark {
  x: 0.6192103028297424
  y: 0.688189685344696
  z: -0.08818458765745163
}
landmark {
  x: 0.6611127853393555
  y: 0.6955099701881409
  z: -0.05048443749547005
}
landmark {
  x: 0.6017453074455261
  y: 0.7119958996772766
  z: -0.07660623639822006
}
landmark {
  x: 0.6376242637634277
  y: 0.7120493650436401
  z: -0.04177974909543991
}
landmark {
  x: 0.7118440866470337
  y: 0.7589071989059448
  z: 0.023515647277235985
}
landmark {
  x: 0.5126743316650391
  y: 0.6043348908424377
  z: -0.21332086622714996
}
landmark {
  x: 0.5123692154884338
  y: 0.5711033940315247
  z: -0.22434364259243011
}
landmark {
  x: 0.8047051429748535
  y: 0.3262219727039337
  z: 0.017701640725135803
}
landmark {
  x: 0.5927931070327759
  y: 0.49367523193359375
  z: -0.041927769780159
}
landmark {
  x: 0.6082389950752258
  y: 0.5855504274368286
  z: -0.12623192369937897
}
landmark {
  x: 0.6092568635940552
  y: 0.5680200457572937
  z: -0.11419199407100677
}
landmark {
  x: 0.8092576861381531
  y: 0.5479861497879028
  z: -0.0014363480731844902
}
landmark {
  x: 0.507614016532898
  y: 0.5319874286651611
  z: -0.20096458494663239
}
landmark {
  x: 0.7031187415122986
  y: 0.2961500585079193
  z: -0.0406055711209774
}
landmark {
  x: 0.760782778263092
  y: 0.3038061857223511
  z: -0.014503950253129005
}
landmark {
  x: 0.8425694704055786
  y: 0.21568787097930908
  z: 0.13522790372371674
}
landmark {
  x: 0.5385988354682922
  y: 0.34032565355300903
  z: -0.06717018038034439
}
landmark {
  x: 0.6027284860610962
  y: 0.3763411343097687
  z: 0.00015584345965180546
}
landmark {
  x: 0.7395942211151123
  y: 0.7144449949264526
  z: 0.02430148981511593
}
landmark {
  x: 0.9136649370193481
  y: 0.7333142757415771
  z: 0.31204113364219666
}
landmark {
  x: 0.5830571055412292
  y: 0.6060899496078491
  z: -0.11036499589681625
}
landmark {
  x: 0.551142156124115
  y: 0.6191096901893616
  z: -0.11160721629858017
}
landmark {
  x: 0.7133134603500366
  y: 0.7155593037605286
  z: 0.027367359027266502
}
landmark {
  x: 0.6942542791366577
  y: 0.7182605862617493
  z: 0.019502172246575356
}
landmark {
  x: 0.7833856344223022
  y: 0.2792634069919586
  z: 0.0030102706514298916
}
landmark {
  x: 0.6151416301727295
  y: 0.5973612666130066
  z: -0.10532160103321075
}
landmark {
  x: 0.6298211812973022
  y: 0.30626943707466125
  z: -0.059158291667699814
}
landmark {
  x: 0.6340736746788025
  y: 0.27706363797187805
  z: -0.06689538061618805
}
landmark {
  x: 0.6665512323379517
  y: 0.14459574222564697
  z: -0.0008641325403004885
}
landmark {
  x: 0.8120237588882446
  y: 0.24646785855293274
  z: 0.06073474511504173
}
landmark {
  x: 0.6456632018089294
  y: 0.20777574181556702
  z: -0.03507254645228386
}
landmark {
  x: 0.830237865447998
  y: 0.3101172149181366
  z: 0.048747479915618896
}
landmark {
  x: 0.8616198301315308
  y: 0.29404959082603455
  z: 0.12381526082754135
}
landmark {
  x: 0.5572219491004944
  y: 0.702317476272583
  z: -0.10649286210536957
}
landmark {
  x: 0.6097401976585388
  y: 0.7025388479232788
  z: -0.08312451094388962
}
landmark {
  x: 0.6494945883750916
  y: 0.7038677930831909
  z: -0.04906775429844856
}
landmark {
  x: 0.5753908157348633
  y: 0.6119466423988342
  z: -0.10112249851226807
}
landmark {
  x: 0.7044703364372253
  y: 0.716620683670044
  z: 0.023401401937007904
}
landmark {
  x: 0.6838364601135254
  y: 0.7439297437667847
  z: 0.006240763235837221
}
landmark {
  x: 0.6874526739120483
  y: 0.7196060419082642
  z: 0.018882296979427338
}
landmark {
  x: 0.5566620826721191
  y: 0.6008176207542419
  z: -0.16197162866592407
}
landmark {
  x: 0.630964994430542
  y: 0.7098885178565979
  z: -0.036925382912158966
}
landmark {
  x: 0.5949345827102661
  y: 0.7120267152786255
  z: -0.062326859682798386
}
landmark {
  x: 0.5532914996147156
  y: 0.7164126634597778
  z: -0.08051052689552307
}
landmark {
  x: 0.5808660387992859
  y: 0.8571116924285889
  z: -0.031214017421007156
}
landmark {
  x: 0.5760382413864136
  y: 0.8280496001243591
  z: -0.06238376349210739
}
landmark {
  x: 0.571864902973175
  y: 0.8095350861549377
  z: -0.06722287088632584
}
landmark {
  x: 0.5670864582061768
  y: 0.7940603494644165
  z: -0.05826123431324959
}
landmark {
  x: 0.5617002248764038
  y: 0.7833285927772522
  z: -0.05180101841688156
}
landmark {
  x: 0.6394295692443848
  y: 0.7579297423362732
  z: -0.014121092855930328
}
landmark {
  x: 0.6471433639526367
  y: 0.7607876062393188
  z: -0.0198096614331007
}
landmark {
  x: 0.6577990651130676
  y: 0.7677105665206909
  z: -0.024386810138821602
}
landmark {
  x: 0.6662388443946838
  y: 0.77713942527771
  z: -0.01703520677983761
}
landmark {
  x: 0.681681215763092
  y: 0.6677284836769104
  z: -0.04070958122611046
}
landmark {
  x: 0.9427776336669922
  y: 0.568905770778656
  z: 0.3679347634315491
}
landmark {
  x: 0.6616540551185608
  y: 0.7447880506515503
  z: 0.00876859575510025
}
landmark {
  x: 0.6731821298599243
  y: 0.7429101467132568
  z: 0.005481828935444355
}
landmark {
  x: 0.5410251021385193
  y: 0.6335012912750244
  z: -0.11239670217037201
}
landmark {
  x: 0.6075263619422913
  y: 0.6156387329101562
  z: -0.07554959505796432
}
landmark {
  x: 0.5455762147903442
  y: 0.6267488598823547
  z: -0.11317762732505798
}
landmark {
  x: 0.6382776498794556
  y: 0.5067825317382812
  z: -0.03245118632912636
}
landmark {
  x: 0.7076281905174255
  y: 0.5195570588111877
  z: -0.026095494627952576
}
landmark {
  x: 0.6209862232208252
  y: 0.5803111791610718
  z: -0.09298138320446014
}
landmark {
  x: 0.7698853611946106
  y: 0.17035594582557678
  z: 0.06264714896678925
}
landmark {
  x: 0.7429320812225342
  y: 0.2153264582157135
  z: 0.008848011493682861
}
landmark {
  x: 0.7155844569206238
  y: 0.26735252141952515
  z: -0.034542590379714966
}
landmark {
  x: 0.6803761720657349
  y: 0.796616792678833
  z: 0.005564854014664888
}
landmark {
  x: 0.545661449432373
  y: 0.29406824707984924
  z: -0.0806029886007309
}
landmark {
  x: 0.5515839457511902
  y: 0.2136651575565338
  z: -0.05765911936759949
}
landmark {
  x: 0.5609226226806641
  y: 0.13781699538230896
  z: -0.034965626895427704
}
landmark {
  x: 0.7430508732795715
  y: 0.4281330108642578
  z: 0.022452041506767273
}
landmark {
  x: 0.844697117805481
  y: 0.4517696499824524
  z: 0.07315035909414291
}
landmark {
  x: 0.5802938938140869
  y: 0.4321155548095703
  z: 0.010565190576016903
}
landmark {
  x: 0.7901830077171326
  y: 0.3747103214263916
  z: 0.04007326439023018
}
landmark {
  x: 0.5615787506103516
  y: 0.4760693609714508
  z: -0.053956516087055206
}
landmark {
  x: 0.5785065293312073
  y: 0.5778008699417114
  z: -0.16147805750370026
}
landmark {
  x: 0.8918441534042358
  y: 0.4735209345817566
  z: 0.11267603933811188
}
landmark {
  x: 0.8184541463851929
  y: 0.4718348979949951
  z: 0.03932137042284012
}
landmark {
  x: 0.768517255783081
  y: 0.4847654700279236
  z: 0.007178612053394318
}
landmark {
  x: 0.6929854154586792
  y: 0.48831745982170105
  z: -0.006274426821619272
}
landmark {
  x: 0.6349441409111023
  y: 0.48269224166870117
  z: -0.011449138633906841
}
landmark {
  x: 0.591370165348053
  y: 0.4720451235771179
  z: -0.01922423392534256
}
landmark {
  x: 0.5042835474014282
  y: 0.4369489550590515
  z: -0.09103569388389587
}
landmark {
  x: 0.8984358310699463
  y: 0.5377718210220337
  z: 0.10063814371824265
}
landmark {
  x: 0.8244988918304443
  y: 0.3667304515838623
  z: 0.05605992674827576
}
landmark {
  x: 0.5030688643455505
  y: 0.62248295545578
  z: -0.19541682302951813
}
landmark {
  x: 0.5940446257591248
  y: 0.5232166647911072
  z: -0.061043690890073776
}
landmark {
  x: 0.9397686123847961
  y: 0.41795897483825684
  z: 0.3411884307861328
}
landmark {
  x: 0.5593149065971375
  y: 0.4572816789150238
  z: -0.02309775911271572
}
landmark {
  x: 0.6279747486114502
  y: 0.5811028480529785
  z: -0.052767619490623474
}
landmark {
  x: 0.7707387208938599
  y: 0.39914965629577637
  z: 0.05126340687274933
}
landmark {
  x: 0.5788552165031433
  y: 0.5565616488456726
  z: -0.14665107429027557
}
landmark {
  x: 0.9362665414810181
  y: 0.6493462920188904
  z: 0.34945130348205566
}
landmark {
  x: 0.580301821231842
  y: 0.4213067293167114
  z: 0.020215287804603577
}
landmark {
  x: 0.5423721075057983
  y: 0.5413954854011536
  z: -0.18082544207572937
}
landmark {
  x: 0.827576756477356
  y: 0.8029569983482361
  z: 0.13183309137821198
}
landmark {
  x: 0.826129674911499
  y: 0.847174346446991
  z: 0.19045203924179077
}
landmark {
  x: 0.9401122331619263
  y: 0.5557646751403809
  z: 0.2341695874929428
}
landmark {
  x: 0.8798336982727051
  y: 0.7598730325698853
  z: 0.16860802471637726
}
landmark {
  x: 0.8934718370437622
  y: 0.35113173723220825
  z: 0.18171405792236328
}
landmark {
  x: 0.6714124083518982
  y: 0.9213674068450928
  z: 0.0226733461022377
}
landmark {
  x: 0.5012639164924622
  y: 0.6293689608573914
  z: -0.14582233130931854
}
landmark {
  x: 0.6347501277923584
  y: 0.5416210889816284
  z: -0.049733251333236694
}
landmark {
  x: 0.8785412311553955
  y: 0.41621002554893494
  z: 0.12761190533638
}
landmark {
  x: 0.7031115293502808
  y: 0.4189247786998749
  z: 0.01105754915624857
}
landmark {
  x: 0.6701067686080933
  y: 0.4246559143066406
  z: 0.003528991714119911
}
landmark {
  x: 0.693998396396637
  y: 0.747170627117157
  z: 0.011281153187155724
}
landmark {
  x: 0.904225766658783
  y: 0.6057406663894653
  z: 0.11217426508665085
}
landmark {
  x: 0.6056143641471863
  y: 0.968888521194458
  z: 0.029406843706965446
}
landmark {
  x: 0.7156542539596558
  y: 0.9208263158798218
  z: 0.10170795768499374
}
landmark {
  x: 0.7654732465744019
  y: 0.8901197910308838
  z: 0.14202842116355896
}
landmark {
  x: 0.6382739543914795
  y: 0.4263345003128052
  z: 0.003714349353685975
}
landmark {
  x: 0.608167827129364
  y: 0.42451217770576477
  z: 0.01025357935577631
}
landmark {
  x: 0.5894688963890076
  y: 0.4233325719833374
  z: 0.01901676133275032
}
landmark {
  x: 0.8561718463897705
  y: 0.35812079906463623
  z: 0.09244342148303986
}
landmark {
  x: 0.610427737236023
  y: 0.40064048767089844
  z: 0.004954040050506592
}
landmark {
  x: 0.6419333219528198
  y: 0.39145931601524353
  z: -0.0035158481914550066
}
landmark {
  x: 0.6723035573959351
  y: 0.38722050189971924
  z: -0.0033594227861613035
}
landmark {
  x: 0.7050137519836426
  y: 0.38829874992370605
  z: 0.004632687196135521
}
landmark {
  x: 0.7280253767967224
  y: 0.3929325044155121
  z: 0.0157832819968462
}
landmark {
  x: 0.922637403011322
  y: 0.3383057117462158
  z: 0.2819123864173889
}
landmark {
  x: 0.7254979610443115
  y: 0.41271597146987915
  z: 0.021820107474923134
}
landmark {
  x: 0.636895477771759
  y: 0.6547951698303223
  z: -0.06942592561244965
}
landmark {
  x: 0.5817331075668335
  y: 0.6023143529891968
  z: -0.12369497120380402
}
landmark {
  x: 0.5518733859062195
  y: 0.6538554430007935
  z: -0.10611211508512497
}
landmark {
  x: 0.7720415592193604
  y: 0.8480919003486633
  z: 0.10292784124612808
}
landmark {
  x: 0.7224955558776855
  y: 0.8843857049942017
  z: 0.06571002304553986
}
landmark {
  x: 0.6056313514709473
  y: 0.949116587638855
  z: -0.012368883937597275
}
landmark {
  x: 0.8745077848434448
  y: 0.7999281883239746
  z: 0.251703143119812
}
landmark {
  x: 0.5891892910003662
  y: 0.4131283760070801
  z: 0.013970219530165195
}
landmark {
  x: 0.5339846611022949
  y: 0.4853363633155823
  z: -0.10536916553974152
}
landmark {
  x: 0.6645001769065857
  y: 0.9482040405273438
  z: 0.05757834389805794
}
landmark {
  x: 0.9371163845062256
  y: 0.6303572654724121
  z: 0.22724252939224243
}
landmark {
  x: 0.6042358875274658
  y: 0.7720387578010559
  z: -0.035778649151325226
}
landmark {
  x: 0.612248957157135
  y: 0.7790498733520508
  z: -0.04143768548965454
}
landmark {
  x: 0.6202538013458252
  y: 0.7911738753318787
  z: -0.04681113734841347
}
landmark {
  x: 0.6267709732055664
  y: 0.8070510029792786
  z: -0.040287576615810394
}
landmark {
  x: 0.6387888193130493
  y: 0.831239640712738
  z: -0.016214247792959213
}
landmark {
  x: 0.6726184487342834
  y: 0.7126936912536621
  z: -0.00914271455258131
}
landmark {
  x: 0.6843034029006958
  y: 0.7081231474876404
  z: -0.008467559702694416
}
landmark {
  x: 0.6945748329162598
  y: 0.7022805213928223
  z: -0.009355273097753525
}
landmark {
  x: 0.7200194597244263
  y: 0.6861039400100708
  z: -0.006583949085325003
}
landmark {
  x: 0.8542413711547852
  y: 0.6205570101737976
  z: 0.038892485201358795
}
landmark {
  x: 0.5316597819328308
  y: 0.45482513308525085
  z: -0.06983059644699097
}
landmark {
  x: 0.5485436916351318
  y: 0.3964008688926697
  z: -0.0035277754068374634
}
landmark {
  x: 0.5719646215438843
  y: 0.40180742740631104
  z: 0.009555423632264137
}
landmark {
  x: 0.6626864671707153
  y: 0.7099243998527527
  z: -0.0063500674441456795
}
landmark {
  x: 0.8646278977394104
  y: 0.7047960758209229
  z: 0.100004643201828
}
landmark {
  x: 0.5146359801292419
  y: 0.39463502168655396
  z: -0.0523100420832634
}
landmark {
  x: 0.6529819369316101
  y: 0.8578700423240662
  z: -0.0028281020931899548
}
landmark {
  x: 0.5042141675949097
  y: 0.4702150225639343
  z: -0.129316508769989
}
landmark {
  x: 0.5673672556877136
  y: 0.5313471555709839
  z: -0.10971097648143768
}
landmark {
  x: 0.5904504060745239
  y: 0.885657787322998
  z: -0.0272047258913517
}
landmark {
  x: 0.7341678738594055
  y: 0.7744194865226746
  z: 0.03884495422244072
}
landmark {
  x: 0.6638760566711426
  y: 0.6042520999908447
  z: -0.04157112166285515
}
landmark {
  x: 0.6976938247680664
  y: 0.8204447031021118
  z: 0.02156316302716732
}
landmark {
  x: 0.7590912580490112
  y: 0.5951480865478516
  z: -0.02994963340461254
}
landmark {
  x: 0.7086312174797058
  y: 0.6325660347938538
  z: -0.028385769575834274
}
landmark {
  x: 0.8003910779953003
  y: 0.6445255279541016
  z: 0.004138569813221693
}
landmark {
  x: 0.6013050675392151
  y: 0.9202969074249268
  z: -0.027641113847494125
}
landmark {
  x: 0.5956445336341858
  y: 0.5460949540138245
  z: -0.08496230840682983
}
landmark {
  x: 0.7609846591949463
  y: 0.8049650192260742
  z: 0.06619337201118469
}
landmark {
  x: 0.713670015335083
  y: 0.8484628796577454
  z: 0.04197729751467705
}
landmark {
  x: 0.7646187543869019
  y: 0.7252383828163147
  z: 0.034109391272068024
}
landmark {
  x: 0.899861216545105
  y: 0.6650470495223999
  z: 0.1269931197166443
}
landmark {
  x: 0.811811089515686
  y: 0.737910807132721
  z: 0.0630282610654831
}
landmark {
  x: 0.9194583296775818
  y: 0.6992863416671753
  z: 0.2123991996049881
}
landmark {
  x: 0.7495314478874207
  y: 0.6656218767166138
  z: -0.005609575193375349
}
landmark {
  x: 0.5628350973129272
  y: 0.5045331716537476
  z: -0.0812133252620697
}
landmark {
  x: 0.5678873062133789
  y: 0.5942215919494629
  z: -0.16940903663635254
}
landmark {
  x: 0.5977365374565125
  y: 0.597644031047821
  z: -0.12983109056949615
}
landmark {
  x: 0.5464522242546082
  y: 0.5735775828361511
  z: -0.194813534617424
}
landmark {
  x: 0.5745513439178467
  y: 0.36621636152267456
  z: -0.015208940021693707
}
landmark {
  x: 0.6327033042907715
  y: 0.341604083776474
  z: -0.023593256250023842
}
landmark {
  x: 0.6859936118125916
  y: 0.3331960439682007
  z: -0.023968541994690895
}
landmark {
  x: 0.7332467436790466
  y: 0.33573848009109497
  z: -0.011009084060788155
}
landmark {
  x: 0.7689438462257385
  y: 0.348848432302475
  z: 0.011183288879692554
}
landmark {
  x: 0.8013812899589539
  y: 0.40506184101104736
  z: 0.06539134681224823
}
landmark {
  x: 0.9359502792358398
  y: 0.48567837476730347
  z: 0.22824189066886902
}
landmark {
  x: 0.7763141393661499
  y: 0.44680675864219666
  z: 0.035724811255931854
}
landmark {
  x: 0.7362672686576843
  y: 0.4579707980155945
  z: 0.01578447036445141
}
landmark {
  x: 0.6839812994003296
  y: 0.4636895954608917
  z: 0.002962634898722172
}
landmark {
  x: 0.6337649822235107
  y: 0.46220487356185913
  z: -0.001766429515555501
}
landmark {
  x: 0.5947312116622925
  y: 0.45500612258911133
  z: -0.0008954411023296416
}
landmark {
  x: 0.5681023001670837
  y: 0.4459196925163269
  z: -0.00256124185398221
}
landmark {
  x: 0.9421491622924805
  y: 0.4931836724281311
  z: 0.3675062358379364
}
landmark {
  x: 0.5998513698577881
  y: 0.6042654514312744
  z: -0.11095136404037476
}
landmark {
  x: 0.5358161330223083
  y: 0.5127532482147217
  z: -0.13812489807605743
}
landmark {
  x: 0.5388860106468201
  y: 0.5981953740119934
  z: -0.19681857526302338
}
landmark {
  x: 0.5211951732635498
  y: 0.6159141659736633
  z: -0.1806589514017105
}
landmark {
  x: 0.5407022833824158
  y: 0.6038531064987183
  z: -0.17878368496894836
}
landmark {
  x: 0.5928868651390076
  y: 0.6134583950042725
  z: -0.09697843343019485
}
landmark {
  x: 0.5145755410194397
  y: 0.6193801164627075
  z: -0.190590038895607
}
landmark {
  x: 0.5130746960639954
  y: 0.6262079477310181
  z: -0.14627952873706818
}
landmark {
  x: 0.5662755370140076
  y: 0.42279717326164246
  z: 0.015079694800078869
}
landmark {
  x: 0.5467715263366699
  y: 0.4294167459011078
  z: -0.006547094322741032
}
landmark {
  x: 0.5349453687667847
  y: 0.4338741898536682
  z: -0.032735634595155716
}
landmark {
  x: 0.7428998947143555
  y: 0.39666956663131714
  z: 0.02821102924644947
}
landmark {
  x: 0.7656942009925842
  y: 0.37987834215164185
  z: 0.02971372753381729
}

Tracking caveats

As of v0.8.7 of Face Mesh, it seems they don't support asymmetric facial expressions (raising only 1 eyebrow or only the right corner of your lips).

Extra info

How to get the landmark coordinates out of the face mesh object: https://stackoverflow.com/questions/67141844/how-do-i-get-the-coordinates-of-face-mash-landmarks-in-mediapipe

Next steps

  1. The creation of a mapping between the 468 3D coordinates (landmarks) output of Face Mesh to AU values (OpenFace uses: 1, 2, 4, 5, 6, 7, 9, 10, 12, 14, 15, 17, 20, 23, 25, 26, 28, and 45).

This might be a research project on itself. Some thoughts:

  • Manual approach: I imagine the cleanest data would be mask generated from a picture of a resting face and a mask generated from a picture with 1 AU at maximum value (maximum muscle contraction of that muscle group). Or a video of a person moving a single AU, but that seems hard to map by hand.
  • Machine learning approach: Generate data by running this solution on videos annotated with AU values. Then creating a ML model that learns the mapping from landmarks to AU values.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions