CHANGELOG.txt

AIRFLOW 1.9.0, 2018-01-02
-------------------------

[AIRFLOW-1525] Fix minor LICENSE and NOTICE issues
[AIRFLOW-XXX] Bump version to 1.9.0
[AIRFLOW-1897][AIRFLOW-1873] Task Logs for running instance not visible in WebUI
[AIRFLOW-XXX] Make sure session is committed
[AIRFLOW-1896] FIX bleach <> html5lib incompatibility
[AIRFLOW-XXX] Fix log handler test
[AIRFLOW-1873] Set TI.try_number to right value depending TI state
[AIRFLOW-1554] Fix wrong DagFileProcessor termination method call
[AIRFLOW-1872] Set context for all handlers including parents
[AIRFLOW-XXX] Add dask lock files to excludes
[AIRFLOW-1839] Fix more bugs in S3Hook boto -> boto3 migration
[AIRFLOW-1795] Correctly call S3Hook after migration to boto3
[AIRFLOW-1813] Bug SSH Operator empty buffer
[AIRFLOW-1794] Remove uses of Exception.message for Python 3
[AIRFLOW-1799] Fix logging line which raises errors
[AIRFLOW-1102] Upgrade Gunicorn >=19.4.0
[AIRFLOW-1756] Fix S3TaskHandler to work with Boto3-based S3Hook
[AIRFLOW-1797] S3Hook.load_string didn't work on Python3
[AIRFLOW-1792] Missing intervals DruidOperator
[AIRFLOW-1789][AIRFLOW-1712] Log SSHOperator stderr to log.warning
[AIRFLOW-1669] Fix Docker and pin Moto to 1.1.19
[AIRFLOW-71] Add support for private Docker images
[AIRFLOW-1779] Add keepalive packets to ssh hook
[AIRFLOW-XXX] Give a clue what the 'ds' variable is
[AIRFLOW-XXX] Correct typos in the faq docs page
[AIRFLOW-1571] Add AWS Lambda Hook
[AIRFLOW-1675] Fix docstrings for API docs
[AIRFLOW-1712][AIRFLOW-756][AIRFLOW-751] Log SSHOperator output
[AIRFLOW-1776] Capture stdout and stderr for logging
[AIRFLOW-1765] Make experimental API securable without needing Kerberos.
[AIRFLOW-1764] The web interface should not use the experimental API
[AIRFLOW-1634] Adds task_concurrency feature
[AIRFLOW-1018] Make processor use logging framework
[AIRFLOW-1695] Add RedshiftHook using boto3
[AIRFLOW-1706] Fix query error for MSSQL backend
[AIRFLOW-1711] Use ldap3 dict for group membership
[AIRFLOW-1757] Add missing options to SparkSubmitOperator
[AIRFLOW-1734][Airflow 1734] Sqoop hook/operator enhancements
[AIRFLOW-1731] Set pythonpath for logging
[AIRFLOW-1641] Handle executor events in the scheduler
[AIRFLOW-1744] Make sure max_tries can be set
[AIRFLOW-1330] Add conn_type argument to CLI when adding connection
[AIRFLOW-926] Fix JDBC Hook
[AIRFLOW-1520] Boto3 S3Hook, S3Log
[AIRFLOW-XXX] Fix DateTime in Tree View
[AIRFLOW-1432] Charts label for Y axis not visible
[AIRFLOW-1743] Verify ldap filters correctly
[AIRFLOW-1745] Restore default signal disposition
[AIRFLOW-1741] Correctly hide second chart on task duration page
[AIRFLOW-1726] Add copy_expert psycopg2 method to PostgresHook
[AIRFLOW-1698] Remove SCHEDULER_RUNS env var in systemd
[AIRFLOW-1694] Stop using itertools.izip
[AIRFLOW-1692] Change test_views filename to support Windows
[AIRFLOW-1722] Fix typo in scheduler autorestart output filename
[AIRFLOW-1691] Add better Google cloud logging documentation
[AIRFLOW-1690] Add detail to gcs error messages
[AIRFLOW-1682] Make S3TaskHandler write to S3 on close
[AIRFLOW-1676] Make GCSTaskHandler write to GCS on close
[AIRFLOW-1635] Allow creating GCP connection without requiring a JSON file
[AIRFLOW-1323] Made Dataproc operator parameter names consistent
[AIRFLOW-1590] fix unused module and variable
[AIRFLOW-988] Fix repeating SLA miss callbacks
[AIRFLOW-1611] Customize logging
[AIRFLOW-1668] Expose keepalives_idle for Postgres connections
[AIRFLOW-1658] Kill Druid task on timeout
[AIRFLOW-1669][AIRFLOW-1368] Fix Docker import
[AIRFLOW-1560] Add AWS DynamoDB hook and operator for inserting batch items
[AIRFLOW-1654] Show tooltips for link icons in DAGs view
[AIRFLOW-1660] Change webpage width to full-width
[AIRFLOW-1664] write file as binary instead of str
[AIRFLOW-1659] Fix invalid obj attribute bug in file_task_handler.py
[AIRFLOW-1650] Fix custom celery config loading
[AIRFLOW-1647] Fix Spark-sql hook
[AIRFLOW-1587] Fix CeleryExecutor import error
[AIRFLOW-1636] Add AWS and EMR connection type
[AIRFLOW-1527] Refactor celery config
[AIRFLOW-1639] Fix Fernet error handling
[AIRFLOW-1628] Fix docstring of sqlsensor
[AIRFLOW-1331] add SparkSubmitOperator option
[AIRFLOW-1627] Only query pool in SubDAG init when necessary
[AIRFLOW-1629] Make extra a textarea in edit connections form
[AIRFLOW-1621] Add tests for server side paging
[AIRFLOW-1519] Add server side paging in DAGs list
[AIRFLOW-289] Make airflow timezone independent
[AIRFLOW-1356] Add `--celery_hostname` to `airflow worker`
[AIRFLOW-1591] Avoid attribute error when rendering logging filename
[AIRFLOW-1031] Replace hard-code to DagRun.ID_PREFIX
[AIRFLOW-1604] Rename logger to log
[AIRFLOW-1512] Add PythonVirtualenvOperator
[AIRFLOW-1617] Fix XSS vulnerability in Variable endpoint
[AIRFLOW-1497] Reset hidden fields when changing connection type
[AIRFLOW-1177] Fix Variable.setdefault w/existing JSON
[AIRFLOW-1600] Fix exception handling in get_fernet
[AIRFLOW-1614] Replace inspect.stack() with sys._getframe()
[AIRFLOW-1613] Make MySqlToGoogleCloudStorageOperator compaitible with python3
[AIRFLOW-1609] Fix gitignore to ignore all venvs
[AIRFLOW-1601] Add configurable task cleanup time
[AIRFLOW-XXX] Bumping Airflow 1.9.0alpha0 version
[AIRFLOW-1608] Handle pending job state in GCP Dataflow hook
[AIRFLOW-1606] Use non static DAG.sync_to_db
[AIRFLOW-1606][Airflow-1606][AIRFLOW-1605][AIRFLOW-160] DAG.sync_to_db is now a normal method
[AIRFLOW-1602] LoggingMixin in DAG class
[AIRFLOW-1593] expose load_string in WasbHook
[AIRFLOW-1597] Add GameWisp as Airflow user
[AIRFLOW-1594] Don't install test packages into python root.[]
[AIRFLOW-1582] Improve logging within Airflow
[AIRFLOW-1476] add INSTALL instruction for source releases
[AIRFLOW-XXX] Save username and password in airflow-pr
[AIRFLOW-1522] Increase text size for var field in variables for MySQL
[AIRFLOW-950] Missing AWS integrations on documentation::integrations
[AIRFLOW-XXX] 1.8.2 release notes
[AIRFLOW-1573] Remove `thrift < 0.10.0` requirement
[AIRFLOW-1584] Remove insecure /headers endpoint
[AIRFLOW-1586] Add mapping for date type to mysql_to_gcs operator
[AIRFLOW-1579] Adds support for jagged rows in Bigquery hook for BQ load jobs
[AIRFLOW-1577] Add token support to DatabricksHook
[AIRFLOW-1580] Error in string formating
[AIRFLOW-1567] Updated docs for Google ML Engine operators/hooks
[AIRFLOW-1574] add 'to' attribute to templated vars of email operator
[AIRFLOW-1572] add carbonite to company list
[AIRFLOW-1568] Fix typo in BigQueryHook
[AIRFLOW-1493][AIRFLOW-XXXX][WIP] fixed dumb thing
[AIRFLOW-1567][Airflow-1567] Renamed cloudml hook and operator to mlengine
[AIRFLOW-1568] Add datastore export/import operators
[AIRFLOW-1564] Use Jinja2 to render logging filename
[AIRFLOW-1562] Spark-sql logging contains deadlock
[AIRFLOW-1556][Airflow 1556] Add support for SQL parameters in BigQueryBaseCursor
[AIRFLOW-108] Add CreditCards.com to companies list
[AIRFLOW-1541] Add channel to template fields of slack_operator
[AIRFLOW-1535] Add service account/scopes in dataproc
[AIRFLOW-1384] Add to README.md CaDC/ARGO
[AIRFLOW-1546] add Zymergen 80to org list in README
[AIRFLOW-1545] Add Nextdoor to companies list
[AIRFLOW-1544] Add DataFox to companies list
[AIRFLOW-1529] Add logic supporting quoted newlines in Google BigQuery load jobs
[AIRFLOW-1521] Fix emplate rendering for BigqueryTableDeleteOperator
[AIRFLOW-1324] Generalize Druid operator and hook
[AIRFLOW-1516] Fix error handling getting fernet
[AIRFLOW-1420][AIRFLOW-1473] Fix deadlock check
[AIRFLOW-1495] Fix migration on index on job_id
[AIRFLOW-1483] Making page size consistent in list
[AIRFLOW-1495] Add TaskInstance index on job_id
[AIRFLOW-855] Replace PickleType with LargeBinary in XCom
[AIRFLOW-1505] Document when Jinja substitution occurs
[AIRFLOW-1504] Log dataproc cluster name
[AIRFLOW-1239] Fix unicode error for logs in base_task_runner
[AIRFLOW-1280] Fix Gantt chart height
[AIRFLOW-1507] Template parameters in file_to_gcs operator
[AIRFLOW-1452] workaround lock on method
[AIRFLOW-1385] Make Airflow task logging configurable
[AIRFLOW-940] Handle error on variable decrypt
[AIRFLOW-1492] Add gauge for task successes/failures
[AIRFLOW-1443] Update Airflow configuration documentation
[AIRFLOW-1486] Unexpected S3 writing log error
[AIRFLOW-1487] Added links to all companies officially using Airflow
[AIRFLOW-1489] Fix typo in BigQueryCheckOperator
[AIRFLOW-1349] Fix backfill to respect limits
[AIRFLOW-1478] Chart owner column should be sortable
[AIRFLOW-1397][AIRFLOW-1] No Last Run column data displyed in Airflow UI 1.8.1
[AIRFLOW-1474] Add dag_id regex feature for `airflow clear` command
[AIRFLOW-1445] Changing HivePartitionSensor UI color to lighter shade
[AIRFLOW-1359] Use default_args in Cloud ML eval
[AIRFLOW-1389] Support createDisposition in BigQueryOperator
[AIRFLOW-1349] Refactor BackfillJob _execute
[AIRFLOW-1459] Fixed broken integration .rst formatting
[AIRFLOW-1448] Revert "Fix cli reading logfile in memory"
[AIRFLOW-1398] Allow ExternalTaskSensor to wait on multiple runs of a task
[AIRFLOW-1399] Fix cli reading logfile in memory
[AIRFLOW-1442] Remove extra space from ignore_all_deps generated command
[AIRFLOW-1438] Change batch size per query in scheduler
[AIRFLOW-1439] Add max billing tier for the BQ Hook and Operator
[AIRFLOW-1437] Modify BigQueryTableDeleteOperator
[Airflow 1332] Split logs based on try number
[AIRFLOW-1385] Create abstraction for Airflow task logging
[AIRFLOW-756][AIRFLOW-751] Replace ssh hook, operator & sftp operator with paramiko based
[AIRFLOW-1393][[AIRFLOW-1393] Enable Py3 tests in contrib/spark_submit_hook[
[AIRFLOW-1345] Dont expire TIs on each scheduler loop
[AIRFLOW-1059] Reset orphaned tasks in batch for scheduler
[AIRFLOW-1255] Fix SparkSubmitHook output deadlock
[AIRFLOW-1359] Add Google CloudML utils for model evaluation
[AIRFLOW-1247] Fix ignore all dependencies argument ignored
[AIRFLOW-1401] Standardize cloud ml operator arguments
[AIRFLOW-1394] Add quote_character param to GCS hook and operator
[AIRFLOW-1402] Cleanup SafeConfigParser DeprecationWarning
[AIRFLOW-1326][[AIRFLOW-1326][AIRFLOW-1184] Don't split argument array -- it's already an array.[
[AIRFLOW-1384] Add ARGO/CaDC as a Airflow user
[AIRFLOW-1357] Fix scheduler zip file support
[AIRFLOW-1382] Add working dir option to DockerOperator
[AIRFLOW-1388] Add Cloud ML Engine operators to integration doc
[AIRFLOW-1387] Add unicode string prefix
[AIRFLOW-1366] Add max_tries to task instance
[AIRFLOW-1300] Enable table creation with TBLPROPERTIES
[AIRFLOW-1271] Add Google CloudML Training Operator
[AIRFLOW-300] Add Google Pubsub hook and operator
[AIRFLOW-1343] Fix dataproc label format
[AIRFLOW-1367] Pass Content-ID To reference inline images in an email, we need to be able to add <img src="cid:{}"/> to the HTML. However currently the Content-ID (cid) is not passed, so we need to add it
[AIRFLOW-1265] Fix celery executor parsing CELERY_SSL_ACTIVE
[AIRFLOW-1272] Google Cloud ML Batch Prediction Operator
[AIRFLOW-1352][AIRFLOW-1335] Revert MemoryHandler change ()[]
[AIRFLOW-1350] Add query_uri param to Hive/SparkSQL DataProc operator
[AIRFLOW-1334] Check if tasks are backfill on scheduler in a join
[AIRFLOW-1343] Add Airflow default label to the dataproc operator
[AIRFLOW-1273] Add Google Cloud ML version and model operators
[AIRFLOW-1273]AIRFLOW-1273] Add Google Cloud ML version and model operators
[AIRFLOW-1321] Fix hidden field key to ignore case
[AIRFLOW-1337] Make log_format key names lowercase
[AIRFLOW-1338][AIRFLOW-782] Add GCP dataflow hook runner change to UPDATING.md
[AIRFLOW-801] Remove outdated docstring on BaseOperator
[AIRFLOW-1344] Fix text encoding bug when reading logs for Python 3.5
[AIRFLOW-1338] Fix incompatible GCP dataflow hook
[AIRFLOW-1333] Enable copy function for Google Cloud Storage Hook
[AIRFLOW-1337] Allow log format customization via airflow.cfg
[AIRFLOW-1320] Update LetsBonus users in README
[AIRFLOW-1335] Use MemoryHandler for buffered logging
[AIRFLOW-1339] Add Drivy to the list of users
[AIRFLOW-1275] Put 'airflow pool' into API
[AIRFLOW-1296] Propagate SKIPPED to all downstream tasks
[AIRFLOW-1317] Fix minor issues in API reference
[AIRFLOW-1308] Disable nanny usage for Dask
[AIRFLOW-1172] Support nth weekday of the month cron expression
[AIRFLOW-936] Add clear/mark success for DAG in the UI
[AIRFLOW-1294] Backfills can loose tasks to execute
[AIRFLOW-1299] Support imageVersion in Google Dataproc cluster
[AIRFLOW-1291] Update NOTICE and LICENSE files to match ASF requirements
[AIRFLOW-1301] Add New Relic to list of companies
[AIRFLOW-1289] Removes restriction on number of scheduler threads
[AIRFLOW-1024] Ignore celery executor errors (#49)
[AIRFLOW-1265] Fix exception while loading celery configurations
[AIRFLOW-1290] set docs author to 'Apache Airflow'
[AIRFLOW-1242] Allowing project_id to have a colon in it.
[AIRFLOW-1282] Fix known event column sorting
[AIRFLOW-1166] Speed up _change_state_for_tis_without_dagrun
[AIRFLOW-1208] Speed-up cli tests
[AIRFLOW-1192] Some enhancements to qubole_operator
[AIRFLOW-1281] Sort variables by key field by default
[AIRFLOW-1277] Forbid KE creation with empty fields
[AIRFLOW-1276] Forbid event creation with end_data earlier than start_date
[AIRFLOW-1263] Dynamic height for charts
[AIRFLOW-1266] Increase width of gantt y axis
[AIRFLOW-1244] Forbid creation of a pool with empty name
[AIRFLOW-1274][HTTPSENSOR] Rename parameter params to data
[AIRFLOW-654] Add SSL Config Option for CeleryExecutor w/ RabbitMQ - Add BROKER_USE_SSL config to give option to send AMQP messages over SSL - Can be set using usual airflow options (e.g. airflow.cfg, env vars, etc.)
[AIRFLOW-1256] Add United Airlines to readme
[AIRFLOW-1251] Add eRevalue to Airflow users
[AIRFLOW-908] Print hostname at the start of cli run
[AIRFLOW-1237] Fix IN-predicate sqlalchemy warning
[AIRFLOW-1243] DAGs table has no default entries to show
[AIRFLOW-1245] Fix random failure in test_trigger_dag_for_date
[AIRFLOW-1248] Fix wrong conf name for worker timeout
[AIRFLOW-1197] : SparkSubmitHook on_kill error
[AIRFLOW-1191] : SparkSubmitHook custom cmd
[AIRFLOW-1234] Cover utils.operator_helpers with UTs
[AIRFLOW-1217] Enable Sqoop logging
[AIRFLOW-645] Support HTTPS connections in HttpHook
[AIRFLOW-1231] Use flask_wtf.CSRFProtect
[AIRFLOW-1232] Remove deprecated readfp warning
[AIRFLOW-1233] Cover utils.json with unit tests
[AIRFLOW-1227] Remove empty column on the Logs view
[AIRFLOW-1226] Remove empty column on the Jobs view
[AIRFLOW-1221] Fix templating bug with DatabricksSubmitRunOperator
[AIRFLOW-1210] Enable DbApiHook unit tests
[AIRFLOW-1199] Fix create modal
[AIRFLOW-1200] Forbid creation of a variable with an empty key
[AIRFLOW-1207] Enable utils.helpers unit tests
[AIRFLOW-1213] Add hcatalog parameters to sqoop
[AIRFLOW-1201] Update deprecated 'nose-parameterized'
[AIRFLOW-1186] Sort dag.get_task_instances by execution_date
[AIRFLOW-1203] Pin Google API client version to fix OAuth issue
[AIRFLOW-1145] Fix closest_date_partition function with before set to True If we're looking for the closest date before, we should take the latest date in the list of date before.
[AIRFLOW-1180] Fix flask-wtf version for test_csrf_rejection
[AIRFLOW-993] Update date inference logic
[AIRFLOW-1170] DbApiHook insert_rows inserts parameters separately
[AIRFLOW-1041] Do not shadow xcom_push method[]
[AIRFLOW-860][AIRFLOW-935] Fix plugin executor import cycle and executor selection
[AIRFLOW-1189] Fix get a DataFrame using BigQueryHook failing
[AIRFLOW-1184] SparkSubmitHook does not split args
[AIRFLOW-1182] SparkSubmitOperator template field
[AIRFLOW-823] Allow specifying execution date in task_info API
[AIRFLOW-1175] Add Pronto Tools to Airflow user list
[AIRFLOW-1150] Fix scripts execution in sparksql hook[]
[AIRFLOW-1141] remove crawl_for_tasks
[AIRFLOW-1193] Add Checkr to company using Airflow
[AIRFLOW-1168] Add closing() to all connections and cursors
[AIRFLOW-1188] Add max_bad_records param to GoogleCloudStorageToBigQueryOperator
[AIRFLOW-1187][AIRFLOW-1185] Fix PyPi package names in documents
[AIRFLOW-1185] Fix PyPi URL in templates
[AIRFLOW-XXX] Updating CHANGELOG, README, and UPDATING after 1.8.1 release
[AIRFLOW-1181] Add delete and list functionality to gcs_hook
[AIRFLOW-1179] Fix Pandas 0.2x breaking Google BigQuery change
[AIRFLOW-1167] Support microseconds in FTPHook modification time
[AIRFLOW-1173] Add Robinhood to who uses Airflow
[AIRFLOW-945][AIRFLOW-941] Remove psycopg2 connection workaround
[AIRFLOW-1140] DatabricksSubmitRunOperator should template the "json" field.
[AIRFLOW-1160] Update Spark parameters for Mesos
[AIRFLOW 1149][AIRFLOW-1149] Allow for custom filters in Jinja2 templates
[AIRFLOW-1036] Randomize exponential backoff
[AIRFLOW-1155] Add Tails.com to community
[AIRFLOW-1142] Do not reset orphaned state for backfills
[AIRFLOW-492] Make sure stat updates cannot fail a task
[AIRFLOW-1119] Fix unload query so headers are on first row[]
[AIRFLOW-1089] Add Spark application arguments
[AIRFLOW-1125] Document encrypted connections
[AIRFLOW-1122] Increase stroke width in UI
[AIRFLOW-1138] Add missing licenses to files in scripts directory
(AIRFLOW-11-38) [AIRFLOW-1136] Capture invalid arguments for Sqoop
[AIRFLOW-1127] Move license notices to LICENSE
[AIRFLOW-1118] Add evo.company to Airflow users
[AIRFLOW-1121][AIRFLOW-1004] Fix `airflow webserver --pid` to write out pid file
[AIRFLOW-1124] Do not set all tasks to scheduled in backfill
[AIRFLOW-1120] Update version view to include Apache prefix
[AIRFLOW-1091] Add script that can compare jira target against merges
[AIRFLOW-1107] Add support for ftps non-default port
[AIRFLOW-1000] Rebrand distribution to Apache Airflow
[AIRFLOW-1094] Run unit tests under contrib in Travis
[AIRFLOW-1112] Log which pool when pool is full in scheduler
[AIRFLOW-1106] Add Groupalia/Letsbonus to the ReadMe
[AIRFLOW-1109] Use kill signal to kill processes and log results
[AIRFLOW-1074] Don't count queued tasks for concurrency limits
[AIRFLOW-1095] Make ldap_auth memberOf come from configuration
[AIRFLOW-1090] Add HBO
[AIRFLOW-1035] Use binary exponential backoff
[AIRFLOW-1081] Improve performance of duration chart
[AIRFLOW-1078] Fix latest_runs endpoint for old flask versions
[AIRFLOW-1085] Enhance the SparkSubmitOperator
[AIRFLOW-1050] Do not count up_for_retry as not ready
[AIRFLOW-1028] Databricks Operator for Airflow
[AIRFLOW-1075] Security docs cleanup
[AIRFLOW-1033][AIFRLOW-1033] Fix ti_deps for no schedule dags
[AIRFLOW-1016] Allow HTTP HEAD request method on HTTPSensor
[AIRFLOW-970] Load latest_runs on homepage async
[AIRFLOW-111] Include queued tasks in scheduler concurrency check
[AIRFLOW-1001] Fix landing times if there is no following schedule
[AIRFLOW-1065] Add functionality for Azure Blob Storage over wasb://
[AIRFLOW-947] Improve exceptions for unavailable Presto cluster
[AIRFLOW-1067] use example.com in examples
[AIRFLOW-1064] Change default sort to job_id for TaskInstanceModelView
[AIRFLOW-1030][AIRFLOW-1] Fix hook import for HttpSensor
[AIRFLOW-1051] Add a test for resetdb to CliTests
[AIRFLOW-1004][AIRFLOW-276] Fix `airflow webserver -D` to run in background
[AIRFLOW-1062] Fix DagRun#find to return correct result
[AIRFLOW-1011] Fix bug in BackfillJob._execute() for SubDAGs
[AIRFLOW-1038] Specify celery serialization options explicitly
[AIRFLOW-1054] Fix broken import in test_dag
[AIRFLOW-1007] Use Jinja sandbox for chart_data endpoint
[AIRFLOW-719] Fix race condition in ShortCircuit, Branch and LatestOnly
[AIRFLOW-1043] Fix doc strings of operators
[AIRFLOW-840] Make ticket renewer python3 compatible
[AIRFLOW-985] Extend the sqoop operator and hook
[AIRFLOW-1034] Make it possible to connect to S3 in sigv4 regions
[AIRFLOW-1045] Make log level configurable via airflow.cfg
[AIRFLOW-1047] Sanitize strings passed to Markup
[AIRFLOW-1040] Fix some small typos in comments and docstrings
[AIRFLOW-1017] get_task_instance shouldn't throw exception when no TI
[AIRFLOW-1006] Add config_templates to MANIFEST
[AIRFLOW-999] Add support for Redis database
[AIRFLOW-1009] Remove SQLOperator from Concepts page
[AIRFLOW-1006] Move config templates to separate files
[AIRFLOW-1005] Improve Airflow startup time
[AIRFLOW-1010] Add convenience script for signing releases
[AIRFLOW-995] Remove reference to actual Airflow issue
[AIRFLOW-681] homepage doc link should pointing to apache repo not airbnb repo
[AIRFLOW-705][AIRFLOW-706] Fix run_command bugs
[AIRFLOW-990] Fix Py27 unicode logging in DockerOperator
[AIRFLOW-963] Fix non-rendered code examples
[AIRFLOW-969] Catch bad python_callable argument
[AIRFLOW-984] Enable subclassing of SubDagOperator
[AIRFLOW-997] Update setup.cfg to point to Apache
[AIRFLOW-994] Add MiNODES to the official airflow user list
[AIRFLOW-995][AIRFLOW-1] Update GitHub PR Template
[AIRFLOW-989] Do not mark dag run successful if unfinished tasks
[AIRFLOW-903] New configuration setting for the default dag view
[AIRFLOW-979] Add GovTech GDS
[AIRFLOW-933] Replace eval with literal_eval to prevent RCE
[AIRFLOW-974] Fix mkdirs race condition
[AIRFLOW-917] Fix formatting of error message
[AIRFLOW-770] Refactor BaseHook so env vars are always read
[AIRFLOW-900] Double trigger should not kill original task instance
[AIRFLOW-900] Fixes bugs in LocalTaskJob for double run protection
[AIRFLOW-932][AIRFLOW-932][AIRFLOW-921][AIRFLOW-910] Do not mark tasks removed when backfilling[
[AIRFLOW-961] run onkill when SIGTERMed
[AIRFLOW-910] Use parallel task execution for backfills
[AIRFLOW-967] Wrap strings in native for py2 ldap compatibility
[AIRFLOW-958] Improve tooltip readability
AIRFLOW-959 Cleanup and reorganize .gitignore
AIRFLOW-960 Add .editorconfig file
[AIRFLOW-931] Do not set QUEUED in TaskInstances
[AIRFLOW-956] Get docs working on readthedocs.org
[AIRFLOW-954] Fix configparser ImportError
[AIRFLOW-941] Use defined parameters for psycopg2
[AIRFLOW-943] Update Digital First Media in users list
[AIRFLOW-942] Add mytaxi to Airflow users
[AIRFLOW-939] add .swp to gitginore
[AIRFLOW-719] Prevent DAGs from ending prematurely
[AIRFLOW-938] Use test for True in task_stats queries
[AIRFLOW-937] Improve performance of task_stats
[AIRFLOW-933] use ast.literal_eval rather eval because ast.literal_eval does not execute input.
[AIRFLOW-925] Revert airflow.hooks change that cherry-pick picked
[AIRFLOW-919] Running tasks with no start date shouldn't break a DAGs UI
[AIRFLOW-802][AIRFLOW-1] Add spark-submit operator/hook
[AIRFLOW-725] Use keyring to store credentials for JIRA
[AIRFLOW-916] Remove deprecated readfp function
[AIRFLOW-911] Add coloring and timing to tests
[AIRFLOW-906] Update Code icon from lightning bolt to file
[AIRFLOW-897] Prevent dagruns from failing with unfinished tasks
[AIRFLOW-896] Remove unicode to 8-bit conversion in BigQueryOperator
[AIRFLOW-899] Tasks in SCHEDULED state should be white in the UI instead of black
[AIRFLOW-895] Address Apache release incompliancies
[AIRFLOW-893][AIRFLOW-510] Fix crashing webservers when a dagrun has no start date
[AIRFLOW-880] Make webserver serve logs in a sane way for remote logs
[AIRFLOW-889] Fix minor error in the docstrings for BaseOperator
[AIRFLOW-809][AIRFLOW-1] Use __eq__ ColumnOperator When Testing Booleans
[AIRFLOW-875] Add template to HttpSensor params
[AIRFLOW-866] Add FTPSensor
[AIRFLOW-881] Check if SubDagOperator is in DAG context manager
[AIRFLOW-885] Add change.org to the users list
[AIRFLOW-836] Use POST and CSRF for state changing endpoints
[AIRFLOW-862] Fix Unit Tests for DaskExecutor
[AIRFLOW-887] Support future v0.16
[AIRFLOW-886] Pass result to post_execute() hook
[AIRFLOW-871] change logging.warn() into warning()
[AIRFLOW-882] Remove unnecessary dag>>op assignment in docs
[AIRFLOW-861] make pickle_info endpoint be login_required
[AIRFLOW-869] Refactor mark success functionality
[AIRFLOW-877] Remove .sql template extension from GCS download operator
[AIRFLOW-826] Add Zendesk hook
[AIRFLOW-842] do not query the DB with an empty IN clause
[AIRFLOW-834] change raise StopIteration into return
[AIRFLOW-832] Let debug server run without SSL
[AIRFLOW-862] Add DaskExecutor
[AIRFLOW-858] Configurable database name for DB operators
[AIRFLOW-863] Example DAGs should have recent start dates
[AIRFLOW-853] use utf8 encoding for stdout line decode
[AIRFLOW-857] Use library assert statements instead of conditionals
[AIRFLOW-856] Make sure execution date is set for local client
[AIRFLOW-854] Add OKI as Airflow user
[AIRFLOW-830][AIRFLOW-829][AIRFLOW-88] Reduce Travis log verbosity
[AIRFLOW-814] Fix Presto*CheckOperator.__init__
[AIRFLOW-793] Enable compressed loading in S3ToHiveTransfer
[AIRFLOW-844] Fix cgroups directory creation
[AIRFLOW-831] Restore import to fix broken tests
[AIRFLOW-794] Access DAGS_FOLDER and SQL_ALCHEMY_CONN exclusively from settings
[AIRFLOW-694] Fix config behaviour for empty envvar
[AIRFLOW-365] Set dag.fileloc explicitly and use for Code view
[AIRFLOW-781] Allow DataFlowOperators to accept jobs stored in GCS

AIRFLOW 1.8.2, 2017-09-04
-------------------------

9a53e66 [AIRFLOW-809][AIRFLOW-1] Use __eq__ ColumnOperator When Testing Booleans
333e0b3 [AIRFLOW-1296] Propagate SKIPPED to all downstream tasks
93825d5 Re-enable caching for hadoop components
33a9dcb Pin Hive and Hadoop to a specific version and create writable warehouse dir
7cff6cd [AIRFLOW-1308] Disable nanny usage for Dask
c6a09c4 Updating CHANGELOG for 1.8.2rc1
570b2ed [AIRFLOW-1294] Backfills can loose tasks to execute
3f48d48 [AIRFLOW-1291] Update NOTICE and LICENSE files to match ASF requirements
e10af9a [AIRFLOW-XXX] Set version to 1.8.2rc1
69bd269 [AIRFLOW-1160] Update Spark parameters for Mesos
9692510 [AIRFLOW 1149][AIRFLOW-1149] Allow for custom filters in Jinja2 templates
6de5330 [AIRFLOW-1119] Fix unload query so headers are on first row[]
b4e9eb8 [AIRFLOW-1089] Add Spark application arguments
a4083f3 [AIRFLOW-1078] Fix latest_runs endpoint for old flask versions
7a02841 [AIRFLOW-1074] Don't count queued tasks for concurrency limits
a2c18a5 [AIRFLOW-1064] Change default sort to job_id for TaskInstanceModelView
d1c64ab [AIRFLOW-1038] Specify celery serialization options explicitly
b4ee88a [AIRFLOW-1036] Randomize exponential backoff
9fca409 [AIRFLOW-993] Update date inference logic
272c2f5 [AIRFLOW-1167] Support microseconds in FTPHook modification time
c7c0b72 [AIRFLOW-1179] Fix Pandas 0.2x breaking Google BigQuery change
acd0166 [AIRFLOW-1263] Dynamic height for charts
7f33f6e [AIRFLOW-1266] Increase width of gantt y axis
fc33c04 [AIRFLOW-1290] set docs author to 'Apache Airflow'
2e9eee3 [AIRFLOW-1282] Fix known event column sorting
2389a8a [AIRFLOW-1166] Speed up _change_state_for_tis_without_dagrun
bf966e6 [AIRFLOW-1192] Some enhancements to qubole_operator
57d5bcd [AIRFLOW-1281] Sort variables by key field by default
802fc15 [AIRFLOW-1244] Forbid creation of a pool with empty name
1232b6a [AIRFLOW-1243] DAGs table has no default entries to show
b0ba3c9 [AIRFLOW-1227] Remove empty column on the Logs view
c406652 [AIRFLOW-1226] Remove empty column on the Jobs view
51a83cc [AIRFLOW-1199] Fix create modal
cac7d4c [AIRFLOW-1200] Forbid creation of a variable with an empty key
5f3ee52 [AIRFLOW-1186] Sort dag.get_task_instances by execution_date
f446c08 [AIRFLOW-1145] Fix closest_date_partition function with before set to True If we're looking for the closest date before, we should take the latest date in the list of date before.
93b8e96 [AIRFLOW-1180] Fix flask-wtf version for test_csrf_rejection
bb56805 [AIRFLOW-1170] DbApiHook insert_rows inserts parameters separately
093b2f0 [AIRFLOW-1150] Fix scripts execution in sparksql hook[]
777f181 [AIRFLOW-1168] Add closing() to all connections and cursors
bc8e912 [AIRFLOW-XXX] Updating CHANGELOG, README, and UPDATING after 1.8.1 release

AIRFLOW 1.8.1, 2017-05-09
-------------------------

[AIRFLOW-1142] SubDAG Tasks Not Executed Even Though All Dependencies Met
[AIRFLOW-1138] Add licenses to files in scripts directory
[AIRFLOW-1127] Move license notices to LICENSE instead of NOTICE
[AIRFLOW-1124] Do not set all task instances to scheduled on backfill
[AIRFLOW-1120] Update version view to include Apache prefix
[AIRFLOW-1062] DagRun#find returns wrong result if external_trigger=False is specified
[AIRFLOW-1054] Fix broken import on test_dag
[AIRFLOW-1050] Retries ignored - regression
[AIRFLOW-1033] TypeError: can't compare datetime.datetime to None
[AIRFLOW-1017] get_task_instance should return None instead of throw an exception for non-existent TIs
[AIRFLOW-1011] Fix bug in BackfillJob._execute() for SubDAGs
[AIRFLOW-1004] `airflow webserver -D` runs in foreground
[AIRFLOW-1001] Landing Time shows "unsupported operand type(s) for -: 'datetime.datetime' and 'NoneType'" on example_subdag_operator
[AIRFLOW-1000] Rebrand to Apache Airflow instead of Airflow
[AIRFLOW-989] Clear Task Regression
[AIRFLOW-974] airflow.util.file mkdir has a race condition
[AIRFLOW-906] Update Code icon from lightning bolt to file
[AIRFLOW-858] Configurable database name for DB operators
[AIRFLOW-853] ssh_execute_operator.py stdout decode default to ASCII
[AIRFLOW-832] Fix debug server
[AIRFLOW-817] Trigger dag fails when using CLI + API
[AIRFLOW-816] Make sure to pull nvd3 from local resources
[AIRFLOW-815] Add previous/next execution dates to available default variables.
[AIRFLOW-813] Fix unterminated unit tests in tests.job (tests/job.py)
[AIRFLOW-812] Scheduler job terminates when there is no dag file
[AIRFLOW-806] UI should properly ignore DAG doc when it is None
[AIRFLOW-794] Consistent access to DAGS_FOLDER and SQL_ALCHEMY_CONN
[AIRFLOW-785] ImportError if cgroupspy is not installed
[AIRFLOW-784] Cannot install with funcsigs > 1.0.0
[AIRFLOW-780] The UI no longer shows broken DAGs
[AIRFLOW-777] dag_is_running is initlialized to True instead of False
[AIRFLOW-719] Skipped operations make DAG finish prematurely
[AIRFLOW-694] Empty env vars do not overwrite non-empty config values
[AIRFLOW-492] Insert into dag_stats table results into failed task while task itself succeeded
[AIRFLOW-139] Executing VACUUM with PostgresOperator
[AIRFLOW-111] DAG concurrency is not honored
[AIRFLOW-88] Improve clarity Travis CI reports

AIRFLOW 1.8.0, 2017-03-12
-------------------------

[AIRFLOW-900] Double trigger should not kill original task instance
[AIRFLOW-900] Fixes bugs in LocalTaskJob for double run protection
[AIRFLOW-932] Do not mark tasks removed when backfilling
[AIRFLOW-961] run onkill when SIGTERMed
[AIRFLOW-910] Use parallel task execution for backfills
[AIRFLOW-967] Wrap strings in native for py2 ldap compatibility
[AIRFLOW-941] Use defined parameters for psycopg2
[AIRFLOW-719] Prevent DAGs from ending prematurely
[AIRFLOW-938] Use test for True in task_stats queries
[AIRFLOW-937] Improve performance of task_stats
[AIRFLOW-933] use ast.literal_eval rather eval because ast.literal_eval does not execute input.
[AIRFLOW-925] Revert airflow.hooks change that cherry-pick picked
[AIRFLOW-919] Running tasks with no start date shouldn't break a DAGs UI
[AIRFLOW-802] Add spark-submit operator/hook
[AIRFLOW-897] Prevent dagruns from failing with unfinished tasks
[AIRFLOW-861] make pickle_info endpoint be login_required
[AIRFLOW-853] use utf8 encoding for stdout line decode
[AIRFLOW-856] Make sure execution date is set for local client
[AIRFLOW-830][AIRFLOW-829][AIRFLOW-88] Reduce Travis log verbosity
[AIRFLOW-831] Restore import to fix broken tests
[AIRFLOW-794] Access DAGS_FOLDER and SQL_ALCHEMY_CONN exclusively from settings
[AIRFLOW-694] Fix config behaviour for empty envvar
[AIRFLOW-365] Set dag.fileloc explicitly and use for Code view
[AIRFLOW-931] Do not set QUEUED in TaskInstances
[AIRFLOW-899] Tasks in SCHEDULED state should be white in the UI instead of black
[AIRFLOW-895] Address Apache release incompliancies
[AIRFLOW-893][AIRFLOW-510] Fix crashing webservers when a dagrun has no start date
[AIRFLOW-793] Enable compressed loading in S3ToHiveTransfer
[AIRFLOW-863] Example DAGs should have recent start dates
[AIRFLOW-869] Refactor mark success functionality
[AIRFLOW-856] Make sure execution date is set for local client
[AIRFLOW-814] Fix Presto*CheckOperator.__init__
[AIRFLOW-844] Fix cgroups directory creation
[AIRFLOW-816] Use static nvd3 and d3
[AIRFLOW-821] Fix py3 compatibility
[AIRFLOW-817] Check for None value of execution_date in endpoint
[AIRFLOW-822] Close db before exception
[AIRFLOW-815] Add prev/next execution dates to template variables
[AIRFLOW-813] Fix unterminated unit tests in SchedulerJobTest
[AIRFLOW-813] Fix unterminated scheduler unit tests
[AIRFLOW-806] UI should properly ignore DAG doc when it is None
[AIRFLOW-812] Fix the scheduler termination bug.
[AIRFLOW-780] Fix dag import errors no longer working
[AIRFLOW-783] Fix py3 incompatibility in BaseTaskRunner
[AIRFLOW-810] Correct down_revision dag_id/state index creation
[AIRFLOW-807] Improve scheduler performance for large DAGs
[AIRFLOW-798] Check return_code before forcing termination
[AIRFLOW-139] Let psycopg2 handle autocommit for PostgresHook
[AIRFLOW-776] Add missing cgroups devel dependency
[AIRFLOW-777] Fix expression to check if a DagRun is in running state
[AIRFLOW-785] Don't import CgroupTaskRunner at global scope
[AIRFLOW-784] Pin funcsigs to 1.0.0
[AIRFLOW-624] Fix setup.py to not import airflow.version as version
[AIRFLOW-779] Task should fail with specific message when deleted
[AIRFLOW-778] Fix completey broken MetastorePartitionSensor
[AIRFLOW-739] Set pickle_info log to debug
[AIRFLOW-771] Make S3 logs append instead of clobber
[AIRFLOW-773] Fix flaky datetime addition in api test
[AIRFLOW-219][AIRFLOW-398] Cgroups + impersonation
[AIRFLOW-683] Add jira hook, operator and sensor
[AIRFLOW-762] Add Google DataProc delete operator
[AIRFLOW-760] Update systemd config
[AIRFLOW-759] Use previous dag_run to verify depend_on_past
[AIRFLOW-757] Set child_process_log_directory default more sensible
[AIRFLOW-692] Open XCom page to super-admins only
[AIRFLOW-737] Fix HDFS Sensor directory.
[AIRFLOW-747] Fix retry_delay not honoured
[AIRFLOW-558] Add Support for dag.catchup=(True|False) Option
[AIRFLOW-489] Allow specifying execution date in trigger_dag API
[AIRFLOW-738] Commit deleted xcom items before insert
[AIRFLOW-729] Add Google Cloud Dataproc cluster creation operator
[AIRFLOW-728] Add Google BigQuery table sensor
[AIRFLOW-741] Log to debug instead of info for app.py
[AIRFLOW-731] Fix period bug for NamedHivePartitionSensor
[AIRFLOW-740] Pin jinja2 to < 2.9.0
[AIRFLOW-663] Improve time units for task performance charts
[AIRFLOW-665] Fix email attachments
[AIRFLOW-734] Fix SMTP auth regression when not using user/pass
[AIRFLOW-702] Fix LDAP Regex Bug
[AIRFLOW-717] Add Cloud Storage updated sensor
[AIRFLOW-695] Retries do not execute because dagrun is in FAILED state
[AIRFLOW-673] Add operational metrics test for SchedulerJob
[AIRFLOW-727] try_number is not increased
[AIRFLOW-715] A more efficient HDFS Sensor:
[AIRFLOW-716] Allow AVRO BigQuery load-job without schema
[AIRFLOW-718] Allow the query URI for DataProc Pig
Log needs to be part of try/catch block
[AIRFLOW-721] Descendant process can disappear before termination
[AIRFLOW-403] Bash operator's kill method leaves underlying processes running
[AIRFLOW-657] Add AutoCommit Parameter for MSSQL
[AIRFLOW-641] Improve pull request instructions
[AIRFLOW-685] Add test for MySqlHook.bulk_load()
[AIRFLOW-686] Match auth backend config section
[AIRFLOW-691] Add SSH KeepAlive option to SSH_hook
[AIRFLOW-709] Use same engine for migrations and reflection
[AIRFLOW-700] Update to reference to web authentication documentation
[AIRFLOW-649] Support non-sched DAGs in LatestOnlyOp
[AIRFLOW-712] Fix AIRFLOW-667 to use proper HTTP error properties
[AIRFLOW-710] Add OneFineStay as official user
[AIRFLOW-703][AIRFLOW-1] Stop Xcom being cleared too early
[AIRFLOW-679] Stop concurrent task instances from running
[AIRFLOW-704][AIRFLOW-1] Fix invalid syntax in BQ hook
[AIRFLOW-667] Handle BigQuery 503 error
[AIRFLOW-680] Disable connection pool for commands
[AIRFLOW-678] Prevent scheduler from double triggering TIs
[AIRFLOW-677] Kill task if it fails to heartbeat
[AIRFLOW-674] Ability to add descriptions for DAGs
[AIRFLOW-682] Bump MAX_PERIODS to make mark_success work for large DAGs
Use jdk selector to set required jdk
[AIRFLOW-647] Restore dag.get_active_runs
[AIRFLOW-662] Change seasons to months in project description
[AIRFLOW-656] Add dag/task/date index to xcom table
[AIRFLOW-658] Improve schema_update_options in GCP
[AIRFLOW-41] Fix pool oversubscription
[AIRFLOW-489] Add API Framework
[AIRFLOW-653] Add some missing endpoint tests
[AIRFLOW-652] Remove obsolete endpoint
[AIRFLOW-345] Add contrib ECSOperator
[AIRFLOW-650] Adding Celect to user list
[AIRFLOW-510] Filter Paused Dags, show Last Run & Trigger Dag
[AIRFLOW-643] Improve date handling for sf_hook
[AIRFLOW-638] Add schema_update_options to GCP ops
[AIRFLOW-640] Install and enable nose-ignore-docstring
[AIRFLOW-639]AIRFLOW-639] Alphasort package names
[AIRFLOW-375] Fix pylint errors
[AIRFLOW-347] Show empty DAG runs in tree view
[AIRFLOW-628] Adding SalesforceHook to contrib/hooks
[AIRFLOW-514] hive hook loads data from pandas DataFrame into hive and infers types
[AIRFLOW-565] Fixes DockerOperator on Python3.x
[AIRFLOW-635] Encryption option for S3 hook
[AIRFLOW-137] Fix max_active_runs on clearing tasks
[AIRFLOW-343] Fix schema plumbing in HiveServer2Hook
[AIRFLOW-130] Fix ssh operator macosx
[AIRFLOW-633] Show TI attributes in TI view
[AIRFLOW-626][AIRFLOW-1] HTML Content does not show up when sending email with attachment
[AIRFLOW-533] Set autocommit via set_autocommit
[AIRFLOW-629] stop pinning lxml
[AIRFLOW-464] Add setdefault method to Variable
[AIRFLOW-626][AIRFLOW-1] HTML Content does not show up when sending email with attachment
[AIRFLOW-591] Add datadog hook & sensor
[AIRFLOW-561] Add RedshiftToS3Transfer operator
[AIRFLOW-570] Pass root to date form on gantt
[AIRFLOW-504] Store fractional seconds in MySQL tables
[AIRFLOW-623] LDAP attributes not always a list
[AIRFLOW-611] source_format in BigQueryBaseCursor
[AIRFLOW-619] Fix exception in Gannt chart
[AIRFLOW-618] Cast DateTimes to avoid sqllite errors
[AIRFLOW-422] Add JSON endpoint for task info
[AIRFLOW-616][AIRFLOW-617] Minor fixes to PR tool UX
[AIRFLOW-179] Fix DbApiHook with non-ASCII chars
[AIRFLOW-566] Add timeout while fetching logs
[AIRFLOW-615] Set graph glyphicon first
[AIRFLOW-609] Add application_name to PostgresHook
[AIRFLOW-604] Revert .first() to .one()
[AIRFLOW-370] Create AirflowConfigException in exceptions.py
[AIRFLOW-582] Fixes TI.get_dagrun filter (removes start_date)
[AIRFLOW-568] Fix double task_stats count if a DagRun is active
[AIRFLOW-585] Fix race condition in backfill execution loop
[AIRFLOW-580] Prevent landscape warning on .format
[AIRFLOW-597] Check if content is None, not false-equivalent
[AIRFLOW-586] test_dag_v1 fails from 0 to 3 a.m.
[AIRFLOW-453] Add XCom Admin Page
[AIRFLOW-588] Add Google Cloud Storage Object sensor[]
[AIRFLOW-592] example_xcom import Error
[AIRFLOW-587] Fix incorrect scope for Google Auth[]
[AIRFLOW-589] Add templatable job_name[]
[AIRFLOW-227] Show running config in config view
[AIRFLOW-319]AIRFLOW-319] xcom push response in HTTP Operator
[AIRFLOW-385] Add symlink to latest scheduler log directory
[AIRFLOW-583] Fix decode error in gcs_to_bq
[AIRFLOW-96] s3_conn_id using environment variable
[AIRFLOW-575] Clarify tutorial and FAQ about `schedule_interval` always inheriting from DAG object
[AIRFLOW-577] Output BigQuery job for improved debugging
[AIRFLOW-560] Get URI & SQLA engine from Connection
[AIRFLOW-518] Require DataProfilingMixin for Variables CRUD
[AIRFLOW-553] Fix load path for filters.js
[AIRFLOW-554] Add Jinja support to Spark-sql
[AIRFLOW-550] Make ssl config check empty string safe
[AIRFLOW-500] Use id for github allowed teams
[AIRFLOW-556] Add UI PR guidelines
[AIRFLOW-358][AIRFLOW-430] Add `connections` cli
[AIRFLOW-548] Load DAGs immediately & continually
[AIRFLOW-539] Updated BQ hook and BQ operator to support Standard SQL.
[AIRFLOW-378] Add string casting to params of spark-sql operator
[AIRFLOW-544] Add Pause/Resume toggle button
[AIRFLOW-333][AIRFLOW-258] Fix non-module plugin components
[AIRFLOW-542] Add tooltip to DAGs links icons
[AIRFLOW-530] Update docs to reflect connection environment var has to be in uppercase
[AIRFLOW-525] Update template_fields in Qubole Op
[AIRFLOW-480] Support binary file download from GCS
[AIRFLOW-198] Implement latest_only_operator
[AIRFLOW-91] Add SSL config option for the webserver
[AIRFLOW-191] Fix connection leak with PostgreSQL backend
[AIRFLOW-512] Fix 'bellow' typo in docs & comments
[AIRFLOW-509][AIRFLOW-1] Create operator to delete tables in BigQuery
[AIRFLOW-498] Remove hard-coded gcp project id
[AIRFLOW-505] Support unicode characters in authors' names
[AIRFLOW-494] Add per-operator success/failure metrics
[AIRFLOW-488] Fix test_simple fail
[AIRFLOW-468] Update Panda requirement to 0.17.1
[AIRFLOW-159] Add cloud integration section + GCP documentation
[AIRFLOW-477][AIRFLOW-478] Restructure security section for clarity
[AIRFLOW-467] Allow defining of project_id in BigQueryHook
[AIRFLOW-483] Change print to logging statement
[AIRFLOW-475] make the segment granularity in Druid hook configurable


AIRFLOW 1.7.2
-------------

[AIRFLOW-463] Link Airflow icon to landing page
[AIRFLOW-149] Task Dependency Engine + Why Isn't My Task Running View
[AIRFLOW-361] Add default failure handler for the Qubole Operator
[AIRFLOW-353] Fix dag run status update failure
[AIRFLOW-447] Store source URIs in Python 3 compatible list
[AIRFLOW-443] Make module names unique when importing
[AIRFLOW-444] Add Google authentication backend
[AIRFLOW-446][AIRFLOW-445] Adds missing dataproc submit options
[AIRFLOW-431] Add CLI for CRUD operations on pools
[AIRFLOW-329] Update Dag Overview Page with Better Status Columns
[AIRFLOW-360] Fix style warnings in models.py
[AIRFLOW-425] Add white fill for null state tasks in tree view.
[AIRFLOW-69] Use dag runs in backfill jobs
[AIRFLOW-415] Make dag_id not found error clearer
[AIRFLOW-416] Use ordinals in README's company list
[AIRFLOW-369] Allow setting default DAG orientation
[AIRFLOW-410] Add 2 Q/A to the FAQ in the docs
[AIRFLOW-407] Add different colors for some sensors
[AIRFLOW-414] Improve error message for missing FERNET_KEY
[AIRFLOW-406] Sphinx/rst fixes
[AIRFLOW-412] Fix lxml dependency
[AIRFLOW-413] Fix unset path bug when backfilling via pickle
[AIRFLOW-78] Airflow clear leaves dag_runs
[AIRFLOW-402] Remove NamedHivePartitionSensor static check, add docs
[AIRFLOW-394] Add an option to the Task Duration graph to show cumulative times
[AIRFLOW-404] Retry download if unpacking fails for hive
[AIRFLOW-276] Gunicorn rolling restart
[AIRFLOW-399] Remove dags/testdruid.py
[AIRFLOW-400] models.py/DAG.set_dag_runs_state() does not correctly set state
[AIRFLOW-395] Fix colon/equal signs typo for resources in default config
[AIRFLOW-397] Documentation: Fix typo "instatiating" to "instantiating"
[AIRFLOW-395] Remove trailing commas from resources in config
[AIRFLOW-388] Add a new chart for Task_Tries for each DAG
[AIRFLOW-322] Fix typo in FAQ section
[AIRFLOW-375] Pylint fixes
limit scope to user email only AIRFLOW-386
[AIRFLOW-383] Cleanup example qubole operator dag
[AIRFLOW-160] Parse DAG files through child processes
[AIRFLOW-381] Manual UI Dag Run creation: require dag_id field
[AIRFLOW-373] Enhance CLI variables functionality
[AIRFLOW-379] Enhance Variables page functionality: import/export variables
[AIRFLOW-331] modify the LDAP authentication config lines in  'Security' sample codes
[AIRFLOW-356][AIRFLOW-355][AIRFLOW-354] Replace nobr, enable DAG only exists locally message, change edit DAG icon
[AIRFLOW-362] Import __future__ division
[AIRFLOW-359] Pin flask-login to 0.2.11
[AIRFLOW-261] Add bcc and cc fields to EmailOperator
[AIRFLOW-348] Fix code style warnings
[AIRFLOW-349] Add metric for number of zombies killed
[AIRFLOW-340] Remove unused dependency on Babel
[AIRFLOW-339]: Ability to pass a flower conf file
[AIRFLOW-341][operators] Add resource requirement attributes to operators
[AIRFLOW-335] Fix simple style errors/warnings
[AIRFLOW-337] Add __repr__ to VariableAccessor and VariableJsonAccessor
[AIRFLOW-334] Fix using undefined variable
[AIRFLOW-315] Fix blank lines code style warnings
[AIRFLOW-306] Add Spark-sql Hook and Operator
[AIRFLOW-327] Add rename method to the FTPHook
[AIRFLOW-321] Fix a wrong code example about tests/dags
[AIRFLOW-316] Always check DB state for Backfill Job execution
[AIRFLOW-264] Adding workload management for Hive
[AIRFLOW-297] support exponential backoff option for retry delay
[AIRFLOW-31][AIRFLOW-200] Add note to updating.md
[AIRFLOW-307] There is no __neq__ python magic method.
[AIRFLOW-309] Add requirements of develop dependencies to docs
[AIRFLOW-307] Rename __neq__ to __ne__ python magic method.
[AIRFLOW-313] Fix code style for sqoop_hook.py
[AIRFLOW-311] Fix wrong path in CONTRIBUTING.md
[AIRFLOW-24] DataFlow Java Operator
[AIRFLOW-308] Add link to refresh DAG within DAG view header
[AIRFLOW-314] Fix BigQuery cursor run_table_upsert method
[AIRFLOW-298] fix incubator diclaimer in docs
[AIRFLOW-284] HiveServer2Hook fix for cursor scope for get_results
[AIRFLOW-260] More graceful exit when issues can't be closed
[AIRFLOW-260] Handle case when no version is found
[AIRFLOW-228] Handle empty version list in PR tool
[AIRFLOW-302] Improve default squash commit message
[AIRFLOW-187] Improve prompt styling
[AIRFLOW-187] Fix typo in argument name
[AIRFLOW-187] Move "Close XXX" message to end of squash commit
[AIRFLOW-247] Add EMR hook, operators and sensors. Add AWS base hook
[AIRFLOW-301] Fix broken unit test
[AIRFLOW-100] Add execution_date_fn to ExternalTaskSensor
[AIRFLOW-282] Remove PR Tool logic that depends on version formatting
[AIRFLOW-291] Add index for state in TI table
[AIRFLOW-269] Add some unit tests for PostgreSQL
[AIRFLOW-296] template_ext is being treated as a string rather than a tuple in qubole operator
[AIRFLOW-286] Improve FTPHook to implement context manager interface
[AIRFLOW-243] Create NamedHivePartitionSensor
[AIRFLOW-246] Improve dag_stats endpoint query
[AIRFLOW-189] Highlighting of Parent/Child nodes in Graphs
[ARFLOW-255] Check dagrun timeout when comparing active runs
[AIRFLOW-281] Add port to mssql_hook
[AIRFLOW-285] Use Airflow 2.0 style imports for all remaining hooks/operators
[AIRFLOW-40] Add LDAP group filtering feature.
[AIRFLOW-277] Multiple deletions does not work in Task Instances view if using SQLite backend
[AIRFLOW-200] Make hook/operator imports lazy, and print proper exceptions
[AIRFLOW-283] Make store_to_xcom_key a templated field in GoogleCloudStorageDownloadOperator
[AIRFLOW-278] Support utf-8 ecoding for SQL
[AIRFLOW-280] clean up tmp druid table no matter if an ingestion job succeeds or not
[AIRFLOW-274] Add XCom functionality to GoogleCloudStorageDownloadOperator
[AIRFLOW-273] Create an svg version of the airflow logo.
[AIRFLOW-275] Update contributing guidelines
[AIRFLOW-244] Modify hive operator to inject analysis data
[AIRFLOW-162] Allow variable to be accessible into templates
[AIRFLOW-248] Add Apache license header to all files
[AIRFLOW-263] Remove temp backtick file
[AIRFLOW-252] Raise Sqlite exceptions when deleting tasks instance in WebUI
[AIRFLOW-180] Fix timeout behavior for sensors
[AIRFLOW-262] Simplify commands in MANIFEST.in
[AIRFLOW-31] Add zope dependency
[AIRFLOW-6] Remove dependency on Highcharts
[AIRFLOW-234] make task that aren't `running` self-terminate
[AIRFLOW-256] Fix test_scheduler_reschedule heartrate
Add Python 3 compatibility fix
[AIRFLOW-31] Use standard imports for hooks/operators
[AIRFLOW-173] Initial implementation of FileSensor
[AIRFLOW-224] Collect orphaned tasks and reschedule them
[AIRFLOW-239] Fix tests indentation
[AIRFLOW-225] Better units for task duration graph
[AIRFLOW-241] Add testing done section to PR template
[AIRFLOW-222] Show duration of task instances in ui
[AIRFLOW-231] Do not eval user input in PrestoHook
[AIRFLOW-216] Add Sqoop Hook and Operator
[AIRFLOW-171] Add upgrade notes on email and S3 to 1.7.1.2
[AIRFLOW-238] Make compatible with flask-admin 1.4.1
[AIRFLOW-230] [HiveServer2Hook] adding multi statements support
[AIRFLOW-142] setup_env.sh doesn't download hive tarball if hdp is specified as distro
[AIRFLOW-223] Make parametrable the IP on which Flower binds to
[AIRFLOW-218] Added option to enable webserver gunicorn access/err logs
[AIRFLOW-213] Add "Closes #X" phrase to commit messages
[AIRFLOW-68] Align start_date with the schedule_interval
[AIRFLOW-9] Improving docs to meet Apache's standards
[AIRFLOW-131] Make XCom.clear more selective
[AIRFLOW-214] Fix occasion of detached taskinstance
[AIRFLOW-206] Add commit to close PR
[AIRFLOW-206] Always load local log files if they exist
[AIRFLOW-211] Fix JIRA "resolve" vs "close" behavior
[AIRFLOW-64] Add note about relative DAGS_FOLDER
[AIRFLOW-114] Sort plugins dropdown
[AIRFLOW-209] Add scheduler tests and improve lineage handling
[AIRFLOW-207] Improve JIRA auth workflow
[AIRFLOW-187] Improve PR tool UX
[AIRFLOW-155] Documentation of Qubole Operator
Optimize and refactor process_dag
[AIRFLOW-185] Handle empty versions list
[AIRFLOW-201] Fix for HiveMetastoreHook + kerberos
[AIRFLOW-202]: Fixes stray print line
[AIRFLOW-196] Fix bug that exception is not handled in HttpSensor
[AIRFLOW-195] : Add toggle support to subdag clearing in the CLI
[AIRFLOW-23] Support for Google Cloud DataProc
[AIRFLOW-25] Configuration for Celery always required
[AIRFLOW-190] Add codecov and remove download count
[AIRFLOW-168] Correct evaluation of @once schedule
[AIRFLOW-183] Fetch log from remote when worker returns 4xx/5xx response
[AIRFLOW-181] Fix failing unpacking of hadoop by redownloading
[AIRFLOW-176] remove unused formatting key
[AIRFLOW-167]: Add dag_state option in cli
[AIRFLOW-178] Fix bug so that zip file is detected in DAG folder
[AIRFLOW-176] Improve PR Tool JIRA workflow
AIRFLOW-45: Support Hidden Airflow Variables
[AIRFLOW-175] Run git-reset before checkout in PR tool
[AIRFLOW-157] Make PR tool Py3-compat; add JIRA command
[AIRFLOW-170] Add missing @apply_defaults


AIRFLOW 1.7.1, 2016-05-19
-------------------------

- Fix : Don't treat premature tasks as could_not_run tasks
- AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/incubator-airflow#1485
- Add logic to lock DB and avoid race condition
- Handle queued tasks from multiple jobs/executors
- AIRFLOW-52 Warn about overwriting tasks in a DAG
- Fix corner case with joining processes/queues (#1473)
- [AIRFLOW-52] Fix bottlenecks when working with many tasks
- Add columns to toggle extra detail in the connection list view.
- Log the number of errors when importing DAGs
- Log dagbag metrics dupplicate messages in queue into Statsd (#1406)
- Clean up issue template (#1419)
- correct missed arg.foreground to arg.daemon in cli
- Reinstate imports for github enterprise auth
- Use os.execvp instead of subprocess.Popen for the webserver
- Revert from using "--foreground" to "--daemon"
- Implement a Cloudant hook
- Add missing args to `airflow clear`
- Fixed a bug in the scheduler: num_runs used where runs intended
- Add multiprocessing support to the scheduler
- Partial fix to make sure next_run_date cannot be None
- Support list/get/set variables in the CLI
- Properly handle BigQuery booleans in BigQuery hook.
- Added the ability to view XCom variables in webserver
- Change DAG.tasks from a list to a dict
- Add support for zipped dags
- Stop creating hook on instantiating of S3 operator
- User subquery in views to find running DAGs
- Prevent DAGs from being reloaded on every scheduler iteration
- Add a missing word to docs
- Document the parameters of `DbApiHook`
- added oracle operator with existing oracle hook
- Add PyOpenSSL to Google cloud gcp_api.
- Remove executor error unit test
- Add DAG inference, deferral, and context manager
- Don't return error when writing files to Google cloud storage.
- Fix GCS logging for gcp_api.
- Ensure attr is in scope for error message
- Fixing misnamed PULL_REQUEST_TEMPLATE
- Extract non_pooled_task_slot_count into a configuration param
- Update plugins.rst for clarity on the example (#1309)
- Fix s3 logging issue
- Add twitter feed example dag
- Github ISSUE_TEMPLATE & PR_TEMPLATE cleanup
- Reduce logger verbosity
- Adding a PR Template
- Add Lucid to list of users
- Fix usage of asciiart
- Use session instead of outdated main_session for are_dependencies_met
- Fix celery flower port allocation
- Fix for missing edit actions due to flask-admin upgrade
- Fix typo in comment in prioritize_queued method
- Add HipchatOperator
- Include all example dags in backfill unit test
- Make sure skipped jobs are actually skipped
- Fixing a broken example dag, example_skip_dag.py
- Add consistent and thorough signal handling and logging
- Allow Operators to specify SKIPPED status internally
- Update docstring for executor trap unit test
- Doc: explain the usage of Jinja templating for templated params
- Don't schedule runs before the DAG's start_date
- Fix infinite retries with pools, with test
- Fix handling of deadlocked jobs
- Show only Airflow's deprecation warnings
- Set DAG_FOLDER for unit tests
- Missing comma in setup.py
- Deprecate *args and **kwargs in BaseOperator
- Raise deep scheduler exceptions to force a process restart.
- Change inconsistent example DAG owners
- Fix module path of send_email_smtp in configuration
- added Gentner Lab to list of users
- Increase timeout time for unit test
- Fix reading strings from conf
- CHORE - Remove Trailing Spaces
- Fix SSHExecuteOperator crash when using a custom ssh port
- Add note about airflow components to template
- Rewrite BackfillJob logic for clarity
- Add unit tests
- Fix miscellaneous bugs and clean up code
- Fix logic for determining DagRun states
- Make SchedulerJob not run EVERY queued task
- Improve BackfillJob handling of queued/deadlocked tasks
- Introduce ignore_depends_on_past parameters
- Use Popen with CeleryExecutor
- Rename user table to users to avoid conflict with postgres
- Beware of negative pool slots.
- Add support for calling_format from boto to S3_Hook
- Add pypi meta data and sync version number
- Set dags_are_paused_at_creation's default value to True
- Resurface S3Log class eaten by rebase/push -f
- Add missing session.commit() at end of initdb
- Validate that subdag tasks have pool slots available, and test
- Use urlparse for remote GCS logs, and add unit tests
- Make webserver worker timeout configurable
- Fixed scheduling for @once interval
- Use psycopg2's API for serializing postgres cell values
- Make the provide_session decorator more robust
- update link to Lyft's website
- use num_shards instead of partitions to be consistent with batch ingestion
- Add documentation links to README
- Update docs with separate configuration section
- Fix airflow.utils deprecation warning code being Python 3 incompatible
- Extract dbapi cell serialization into its own method
- Set Postgres autocommit as supported only if server version is < 7.4
- Use refactored utils module in unit test imports
- Add changelog for 1.7.0
- Use LocalExecutor on Travis if possible
- remove unused logging,errno, MiniHiveCluster imports
- remove extra import of logging lib
- Fix required gcloud version
- Refactoring utils into smaller submodules
- Properly measure number of task retry attempts
- Add function to get configuration as dict, plus unit tests
- Merge branch 'master' into hivemeta_sasl
- Add wiki link to README.md
- [hotfix] make email.Utils > email.utils for py3
- Add the missing "Date" header to the warning e-mails
- Add the missing "Date" header to the warning e-mails
- Check name of SubDag class instead of class itself
- [hotfix] removing repo_token from .coveralls.yml
- Set the service_name in coverals.yml
- Fixes #1223
- Update Airflow docs for remote logging
- Add unit tests for trapping Executor errors
- Make sure Executors properly trap errors
- Fix HttpOpSensorTest to use fake resquest session
- Linting
- Add an example on pool usage in the documentation
- Add two methods to bigquery hook's base cursor: run_table_upsert, which adds a table or updates an existing table; and run_grant_dataset_view_access, which grants view access to a given dataset for a given table.
- Tasks references upstream and downstream tasks using strings instead of references
- Fix typos in models.py
- Fix broken links in documentation
- [hotfix] fixing the Scheduler CLI to make dag_id optional
- Update link to Common Pitfalls wiki page in README
- Allow disabling periodic committing when inserting rows with DbApiHook
- added Glassdoor to "who uses airflow"
- Fix typo preventing from launching webserver
- Documentation badge
- Fixing ISSUE_TEMPLATE name to include .md suffix
- Adding an ISSUE_TEMPLATE to ensure that issues are adequately defined
- Linting & debugging
- Refactoring the CLI to be data-driven
- Updating the Bug Reporting protocol in the Contributing.md file
- Fixing the docs
- clean up references to old session
- remove session reference
- resolve conflict
- clear xcom data when task instance starts
- replace main_session with @provide_session
- Add extras to installation.rst
- Changes to Contributing to reflect more closely the current state of development.
- Modifying README to link to the wiki committer list
- docs: fixes a spelling mistake in default config
- Set killMode to 'control-group' for webservice.service
- Set KillMode to 'control-group' for worker.service
- Linting
- Fix WebHdfsSensor
- Adding more licenses to pass checks
- fixing landscape's config
- [hotfix] typo that made it in master
- [hotfix] fixing landscape requirement detection
- Make testing on hive conditional
- Merge remote-tracking branch 'upstream/master' into minicluster
- Update README.md
- Throwing in a few license to pass the build
- Adding a reqs.txt for landscape.io
- Pointing to a reqs file
- Some linting
- Adding a .landscape.yml file
- badge for pypi version
- Add license and ignore for sql and csv
- Use correct connection id
- Use correct table name
- Provide data for ci tests
- new badge for showing staleness of reqs
- removing requirements.txt as it is uni-dimensional
- Make it work on py3
- Remove decode for logging
- Also keep py2 compatible
- More py3 fixes
- Convert to bytes for py3 compat
- Make sure to be py3 compatible
- Use unicodecsv to make it py3 compatible
- Replace tab with spaces Remove unused import
- Merge remote-tracking branch 'upstream/master'
- Support decimal types in MySQL to GCS
- Make sure to write binary as string can be unicode
- Ignore metastore
- More impyla fixes
- Test HivemetaStore if python 2
- Allow users to set hdfs_namenode_principal in HDFSHook config
- Add tests for Hiveserver2 and fix some issues from impyla
- Merge branch 'impyla' into minicluster
- This patch allows for testing of hive operators and hooks. Sasl is used (NoSasl in connection string is not possible). Tests have been adjusted.
- Treat SKIPPED and SUCCESS the same way when evaluating depends_on_past=True
- fix bigquery hook
- version cap for gcp_api
- Fix typo when returning VerticaHook
- Adding fernet key to use it as part of stdout commands
- Adding support for ssl parameters.  (picking up from jthomas123)
- more detail in error message.
- make sure paths don't conflict bc of trailing /
- change gcs_hook to self.hook
- refactor remote log read/write and add GCS support
- Only use multipart upload in S3Hook if file is large enough
- Merge branch 'airbnb/master'
- Add GSSAPI SASL to HiveMetaStoreHook.
- Add warning for deprecated setting
- Use kerberos_service_name = 'hive' as standard instead of 'impala'.
- Use GSSAPI instead of KERBEROS and provide backwards compatibility
- ISSUE-1123 Use impyla instead of pyhs2
- set celery_executor to use queue name as exchange