Migration (#1)

* beginning migration of hippyflow

* updating collectives, migrating away from the open PR brach of hippylib for collectives, moved those files here so now I am using hippylib master, adding generalized EVP for the active subspace

* working on updating active subspace and kle

* updating active subspace and driver

* updating

* updating readme and data generators

* adding ice and confusion models

* updating slightly for ccgo machines

* missing hdf5s

* adding confusion neural network training driver

* updating ice problem setup, parallel runs are no go for larger 3d meshes I guess

* updating

* updating helmholtz is crashing my computer

* updated some things want to check in

* this is the setup I used that gets 3% margin of victory

* updating gitignore

* updating gitignore

* updating the repo

* removing ice_flow for now

* adding notebook for plotting eigenvectors

* updating the repo

* updating

* updating

* udapting

* readme rendering

* readme rendering

* readme rendering

* updating install markdown

* adding before removing

* updating with licences

* updating header for copyright

* updating copyright

* adding version

* wrong version in version

* updating hippylib files

* adding some doc strings

* adding more docstrings

* updating

* adding helmholtz neural network training

* updating for the night

* updatings

* updating markdown files

* install readme

* install readme

* install readme

* install readme

Author: tomoleary

.gitignore

@@ -3,6 +3,10 @@ __pycache__/
 *.py[cod]
 *$py.class
 
+# My custom ignores
+*logging*
+*.DS_Store*
+
 # C extensions
 *.so
 

COPYRIGHT

@@ -0,0 +1,35 @@
+# This file is part of the hIPPYflow library. 
+#
+# hIPPYflow is free software; you can redistribute it and/or modify it under the
+# terms of the GNU General Public License (as published by the Free
+# Software Foundation) version 2.0 dated June 1991.
+
+COPYRIGHT
+
+The following copyright applies to each file in the hIPPYlib
+distribution, unless otherwise stated in the file:
+
+Copyright (c) 2020-, The University of Texas at Austin 
+ & Washington University in St. Louis.
+
+LICENSE
+
+hIPPYflow is a free software; you can redistribute it and/or modify it
+under the terms of the GNU General Public License (as published by the
+Free Software Foundation) version 2.0 dated June 1991.
+
+hIPPYflow is distributed in the hope that it will be useful, but
+WITHOUT ANY WARRANTY; without even the IMPLIED WARRANTY OF
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the terms
+and conditions of the GNU Lesser General Public License for more
+details.
+
+You should have received a copy of the GNU Lesser General Public
+License along with this library (file LICENSE); if not, write to the
+Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
+Boston, MA 02110-1301 USA.
+
+
+CONTACT INFORMATION
+
+tom.olearyroseberry@utexas.edu

INSTALL.md

@@ -0,0 +1,53 @@
+		Dimension reduced surrogate construction for parametric PDE maps
+
+		      ___                       ___         ___               
+		     /__/\        ___          /  /\       /  /\        ___   
+		     \  \:\      /  /\        /  /::\     /  /::\      /__/|  
+		      \__\:\    /  /:/       /  /:/\:\   /  /:/\:\    |  |:|  
+		  ___ /  /::\  /__/::\      /  /:/~/:/  /  /:/~/:/    |  |:|  
+		 /__/\  /:/\:\ \__\/\:\__  /__/:/ /:/  /__/:/ /:/   __|__|:|  
+		 \  \:\/:/__\/    \  \:\/\ \  \:\/:/   \  \:\/:/   /__/::::\  
+		  \  \::/          \__\::/  \  \::/     \  \::/       ~\~~\:\ 
+		   \  \:\          /__/:/    \  \:\      \  \:\         \  \:\
+		    \  \:\         \__\/      \  \:\      \  \:\         \__\/
+		     \__\/                     \__\/       \__\/              
+                                                              
+
+			      ___                       ___           ___     
+			     /  /\                     /  /\         /__/\    
+			    /  /:/_                   /  /::\       _\_ \:\   
+			   /  /:/ /\  ___     ___    /  /:/\:\     /__/\ \:\  
+			  /  /:/ /:/ /__/\   /  /\  /  /:/  \:\   _\_ \:\ \:\ 
+			 /__/:/ /:/  \  \:\ /  /:/ /__/:/ \__\:\ /__/\ \:\ \:\
+			 \  \:\/:/    \  \:\  /:/  \  \:\ /  /:/ \  \:\ \:\/:/
+			  \  \::/      \  \:\/:/    \  \:\  /:/   \  \:\ \::/ 
+			   \  \:\       \  \::/      \  \:\/:/     \  \:\/:/  
+			    \  \:\       \__\/        \  \::/       \  \::/   
+			     \__\/                     \__\/         \__\/    
+			                                                      
+                                                      
+
+* PDE data generation is handled by `FEniCS` and `hIPPYlib`. For this [`hIPPYlib`](https://github.com/hippylib/hippylib) must be installed. 
+
+With conda
+
+* `conda create -n hippyflow -c uvilla -c conda-forge fenics==2019.1.0 tensorflow matplotlib scipy jupyter mpi4py keras`
+
+* `pip install pympler` pympler is used to check memory usage during subspace creation
+
+Assumes that the environmental variables `HIPPYLIB_PATH` and `HIPPYFLOW_PATH` have been set.
+
+* `export HIPPYLIB_PATH=path/to/hippylib`
+* `export HIPPYFLOW_PATH=path/to/hippyflow`
+
+
+
+Neural network training is handled by `keras` / second order methods in hessianlearn
+
+When using second order optimizers, the code assumes that the environmental variable `HESSIANLEARN_PATH` has been set to the path to [`hessianlearn`](https://github.com/tomoleary/hessianlearn).
+
+* `export HESSIANLEARN_PATH=path/to/hessianlearn`
+
+In some cases (such as on Unix clusters) the dependencies for `hIPPYlib` and `tensorflow` clash. In this case different conda environments may be needed for the data generation phase and the neural network training phase. These two phases can be easily de-coupled in this case. 
+
+On Mac, there has been no issue creating a singular conda environment to handle all dependencies.

LICENSE

@@ -1 +1,49 @@
-# hippyflow
+		Dimension reduced surrogate construction for parametric PDE maps
+
+		      ___                       ___         ___               
+		     /__/\        ___          /  /\       /  /\        ___   
+		     \  \:\      /  /\        /  /::\     /  /::\      /__/|  
+		      \__\:\    /  /:/       /  /:/\:\   /  /:/\:\    |  |:|  
+		  ___ /  /::\  /__/::\      /  /:/~/:/  /  /:/~/:/    |  |:|  
+		 /__/\  /:/\:\ \__\/\:\__  /__/:/ /:/  /__/:/ /:/   __|__|:|  
+		 \  \:\/:/__\/    \  \:\/\ \  \:\/:/   \  \:\/:/   /__/::::\  
+		  \  \::/          \__\::/  \  \::/     \  \::/       ~\~~\:\ 
+		   \  \:\          /__/:/    \  \:\      \  \:\         \  \:\
+		    \  \:\         \__\/      \  \:\      \  \:\         \__\/
+		     \__\/                     \__\/       \__\/              
+                                                              
+
+			      ___                       ___           ___     
+			     /  /\                     /  /\         /__/\    
+			    /  /:/_                   /  /::\       _\_ \:\   
+			   /  /:/ /\  ___     ___    /  /:/\:\     /__/\ \:\  
+			  /  /:/ /:/ /__/\   /  /\  /  /:/  \:\   _\_ \:\ \:\ 
+			 /__/:/ /:/  \  \:\ /  /:/ /__/:/ \__\:\ /__/\ \:\ \:\
+			 \  \:\/:/    \  \:\  /:/  \  \:\ /  /:/ \  \:\ \:\/:/
+			  \  \::/      \  \:\/:/    \  \:\  /:/   \  \:\ \::/ 
+			   \  \:\       \  \::/      \  \:\/:/     \  \:\/:/  
+			    \  \:\       \__\/        \  \::/       \  \::/   
+			     \__\/                     \__\/         \__\/    
+			                                                      
+                                                      
+
+* This code is used to construct dimension reduced neural network surrogeates for parametric mappings governed by PDEs
+
+* [`hIPPYlib`](https://github.com/hippylib/hippylib) is used for the data generation and construction of model based projectors
+
+* `Tensorflow` and `keras` are used for the construction of neural network surrogates
+
+* [`hessianlearn`](https://github.com/tomoleary/hessianlearn) is used for second order optimization of keras neural network models.
+
+
+## Model Based Projectors:
+
+* `hIPPYflow` implements software infrastructure for input and output dimension reduction strategies for parametric mappings governed by PDEs. Given a parametric PDE Variational Problem implemented in `hIPPYlib` (using `FEniCS` for finite element representation), and a PDE observable, this code automates the construction of dominant subspaces of the input and output for these mappings. `hIPPYflow` implements both active subspace (AS) and Karhunen Loeve expansion (KLE) for input dimension reduction. `hIPPYflow` implements proper orthogonal decomposition (POD) for output dimension reduction.
+
+* These constructs also implement the generation of training data to be used in surrogate construction, as well as projection error tests that exemplify how good the different model projectors are at capturing key information, and help to detect the "intrinsic dimensionality" of the mappings from inputs to outputs.
+
+## Dimension Reduced Neural Network Strategies
+
+* Given information about dominant subspaces of the input and output spaces for the parametric mappings, `hIPPYflow` implements dimension reduced neural network surrogates. These surrogates allow for parsimonious representations of input-output mappings that can achieve good accuracy for very few training data. Few data is a key feature of many high dimensional PDE based inference problems. 
+
+* Neural network models are implemented in `keras`. Training can be handled directly by `keras`, or using second order optimizers implemented in `hessianlearn`.

README.md

@@ -0,0 +1 @@
+__pycache__/*

applications/confusion/.gitignore

@@ -0,0 +1,150 @@
+# Copyright (c) 2020, The University of Texas at Austin 
+# & Washington University in St. Louis.
+#
+# All Rights reserved.
+# See file COPYRIGHT for details.
+#
+# This file is part of the hIPPYflow package. For more information see
+# https://github.com/hippylib/hippyflow/
+#
+# hIPPYflow is free software; you can redistribute it and/or modify it under the
+# terms of the GNU General Public License (as published by the Free
+# Software Foundation) version 2.0 dated June 1991.
+
+
+import dolfin as dl
+import numpy as np
+import ufl
+
+path_to_hippylib = '../../hippylib/'
+import sys, os
+sys.path.append( os.environ.get('HIPPYLIB_PATH',path_to_hippylib))
+from hippylib import *
+
+sys.path.append( os.environ.get('HIPPYFLOW_PATH'))
+from hippyflow import LinearStateObservable
+
+def confusion_linear_observable(mesh,sqrt_n_obs = 10,output_folder ='confusion_setup/',\
+									 verbose = False,seed = 0):
+	class confusion_varf:
+		def __init__(self,Vh,save_fields = False,output_folder = 'confusion_setup/'):
+			'''
+		
+			'''
+			self.Vh = Vh
+			# Gaussian blob for the right hand side
+			self.f = dl.interpolate( dl.Expression('max(0.5,exp(-25*(pow(x[0]-0.7,2) +  pow(x[1]-0.7,2))))',degree=5), Vh[STATE])
+			# Compute velocity field
+			self.v = self.computeVelocityField(Vh[STATE].mesh())
+			if save_fields:
+				if not os.path.isdir(output_folder):
+					os.mkdir(output_folder)
+				f_pvd = dl.File(output_folder+'f_blob.pvd')
+				f_pvd << self.f
+				v_pvd = dl.File(output_folder+'v_sol.pvd')
+				v_pvd << self.v
+			# Constant coefficients for the PDE
+			self.c = dl.Constant(1.0)
+			self.k = dl.Constant(0.01)
+
+
+		def computeVelocityField(self,mesh):
+			def v_boundary(x,on_boundary):
+				return on_boundary
+
+			def q_boundary(x,on_boundary):
+				return x[0] < dl.DOLFIN_EPS and x[1] < dl.DOLFIN_EPS
+			
+			Xh = dl.VectorFunctionSpace(mesh,'Lagrange', 2)
+			Wh = dl.FunctionSpace(mesh, 'Lagrange', 1)
+			mixed_element = ufl.MixedElement([Xh.ufl_element(), Wh.ufl_element()])
+			XW = dl.FunctionSpace(mesh, mixed_element)
+
+			Re = dl.Constant(1e2)
+
+			g = dl.Expression(('0.0','(x[0] < 1e-14) - (x[0] > 1 - 1e-14)'), degree=1)
+			bc1 = dl.DirichletBC(XW.sub(0), g, v_boundary)
+			bc2 = dl.DirichletBC(XW.sub(1), dl.Constant(0), q_boundary, 'pointwise')
+			bcs = [bc1, bc2]
+
+			vq = dl.Function(XW)
+			(v,q) = ufl.split(vq)
+			(v_test, q_test) = dl.TestFunctions (XW)
+
+			def strain(v):
+				return ufl.sym(ufl.grad(v))
+
+			F = ( (2./Re)*ufl.inner(strain(v),strain(v_test))+ ufl.inner (ufl.nabla_grad(v)*v, v_test)
+				   - (q * ufl.div(v_test)) + ( ufl.div(v) * q_test) ) * ufl.dx
+
+			dl.solve(F == 0, vq, bcs, solver_parameters={"newton_solver":
+												 {"relative_tolerance":1e-4, "maximum_iterations":150}})
+			return vq.split()[0]
+
+
+		def __call__(self,u,m,p):
+			'''
+				Return the variational form of the PDE
+				Inputs
+					-u: state variable
+					-m: model parameter
+					-p: adjoint variable
+				Outputs:
+					Variational form of the PDE
+			'''
+			h = dl.CellDiameter(Vh[STATE].mesh())
+			v_norm = dl.sqrt( dl.dot(self.v,self.v) + 1e-6)
+
+			return (h/v_norm)*dl.dot( self.v, dl.nabla_grad(u)) * dl.dot( self.v, dl.nabla_grad(p)) * dl.dx\
+				+ self.k*dl.inner(dl.nabla_grad(u), dl.nabla_grad(p))*dl.dx \
+			   + dl.inner(dl.nabla_grad(u), self.v*p)*dl.dx \
+			   + self.c*dl.exp(m)*u*u*u*p*dl.dx \
+			   - self.f*p*dl.dx	 
+
+
+
+	def u_boundary(x, on_boundary):
+		return on_boundary
+
+	########################################################################
+
+	# Construct the linear observable
+
+	# Define the function spaces
+	Vh1 = dl.FunctionSpace(mesh, 'Lagrange', 1)
+	Vh = [Vh1, Vh1, Vh1]
+	if verbose:
+		print( "Number of dofs: STATE={0}, PARAMETER={1}, ADJOINT={2}".format(Vh[STATE].dim(), Vh[PARAMETER].dim(), Vh[ADJOINT].dim()) )
+
+	# Define points for the observable
+	# Turn this into a meshgrid?
+	x_targets = np.linspace(0.6,0.8,sqrt_n_obs)
+	y_targets = np.linspace(0.6,0.8,sqrt_n_obs)
+	targets = []
+	for xi in x_targets:
+		for yi in y_targets:
+			targets.append((xi,yi))
+	targets = np.array(targets)
+	if verbose:
+		print( "Number of observation points: {0}".format(targets.shape[0]) )
+
+	# Define Dirichlet boundary conditions
+	u_bdr = dl.Constant(0.0)
+	u_bdr0 = dl.Constant(0.0)
+	bc = dl.DirichletBC(Vh[STATE], u_bdr, u_boundary)
+	bc0 = dl.DirichletBC(Vh[STATE], u_bdr0, u_boundary)
+
+	
+	m0 = dl.interpolate(dl.Constant(0.0), Vh[PARAMETER]).vector()
+	param_dimension = m0.get_local().shape[0]
+	m0.set_local(np.random.randn(param_dimension))
+
+	varf_handler = confusion_varf(Vh, output_folder = output_folder)
+
+	pde = PDEVariationalProblem(Vh, varf_handler, bc, bc0, is_fwd_linear=False)
+
+	B = assemblePointwiseObservation(Vh[STATE], targets)
+
+	observable = LinearStateObservable(pde,B)
+
+	return observable