Event async #290

Provide a new min_connection_timeout hook that modules enforcing a dynamic connection timeout (e.g. mod_reqtimeout) should use to inform ap_get_connection_timeout() users about the current timeout being applied. Track the current timeout enforced by mod_reqtimeout in its context and implement the min_connection_timeout to return it. * include/ap_mmn.h(): Minor bump for min_connection_timeout and ap_get_connection_timeout(). * include/http_connection.h(): Declare min_connection_timeout and ap_get_connection_timeout(). * server/connection.c(): Implement min_connection_timeout and ap_get_connection_timeout(). * modules/filters/mod_reqtimeout.c(struct reqtimeout_stage_t): Add server_timeout as the timeout defined for the server at the current stage. * modules/filters/mod_reqtimeout.c(struct reqtimeout_con_cfg): Add time_left as the dynamic timeout enforced by mod_reqtimeout at the current stage. * modules/filters/mod_reqtimeout.c(check_time_left): Store the computed time_left in the reqtimeout_con_cfg, and set the socket timeout there (returning an error which will be caught if that fails). * modules/filters/mod_reqtimeout.c(extend_timeout): Update time_left in the reqtimeout_con_cfg per the time taken by the last read. * modules/filters/mod_reqtimeout.c(reqtimeout_filter): Remove the special path for APR_NONBLOCK_READ or AP_MODE_EATCRLF, it does the exact same thing than the !(AP_MODE_GETLINE && APR_BLOCK_READ) one. * modules/filters/mod_reqtimeout.c(reqtimeout_init, reqtimeout_before_header, reqtimeout_before_body, INIT_STAGE): Set the server_timeout in the current stage. * modules/filters/mod_reqtimeout.c(reqtimeout_min_timeout): The new hook implementation.

If ap_run_process_connection() returns AGAIN and the connection timeout as returned by ap_get_connection_timeout() is different than the rl_q timeout, use a timer event rather than the rl_q to keep track of the idle connection. * server/mpm_fdqueue.h(truct timer_event_t): Add the "timeout" field to store the timeout of the timer, recomputing it from "when" would require to call apr_time_now() otherwise. * server/mpm/event/event.c(): #define TIMER_MIN_TIMEOUT as the minimal timer event's timeout, to prevent the events from firing before the sockets are added to the pollset. Currently set to 50ms (an arbitrary value..). * server/mpm/event/event.c(struct event_conn_state_t): Add the timer_event_t *te field as an alternative to the q. * server/mpm/event/event.c(struct event_srv_cfg_s): Add the server_rec *s field to backref the server_rec and easily pass cs->sc->s to ap_get_connection_timeout(). * server/mpm/event/event.c(pollset_add_at, pollset_del_at): If the connection is attached to a timer event, log a "t" instead of a "q" and the timer's timeout instead of the q's. * server/mpm/event/event.c(process_socket): If ap_get_connection_timeout() is different than the rl_q timeout, acquire a timer event and associate it with the conn_state. A timer event associated with a conn_state has a NULL callback (cbfn). * server/mpm/event/event.c(event_get_timer_event): Set the given timeout to the ->timeout field. * server/mpm/event/event.c(event_register_timed_callback_ex, event_register_poll_callback_ex): Return APR_EINVAL if the given callbacks are NULL, this is reserved for conn_state timers now. Since it would have crashed at some point to pass NULL callbacks before, it's not really an API change. * server/mpm/event/event.c(listener_thread): Fix the poll() timeout set from timers_next_expiry which should be taken into account whether it expired or not. When a conn_state timer fires/expires, remove it from the pollset and abort the connection (with APLOG_INFO). When a conn_state timer is polled, cancel the timer.

…vailable. Connections that need processing while no worker thread is idle are put in a pending_q, the next finishing workers will process them. Listening sockets are disabled in the meantime, when some workers become idle again they will test for should_enable_listensocks() and re-enable them. * server/mpm/event/event.c(): Define MAX_SECS_PENDING as the pending_q timeout. Currently set to 30s (arbitrary value..). * server/mpm/event/event.c(struct timeout_queue): Add the pending_q head pointer. * server/mpm/event/event.c(disable_listensocks, enable_listensocks): Return whether the listening sockets were or not already dis/enabled. * server/mpm/event/event.c(make_conn_state): New helper to allocate and minimally initialize a conn_state since it can now happen from multiple places. The connection_count is incremented there. * server/mpm/event/event.c(process_socket): A new connection is now !cs or !cs->c, since a conn_state can be created earlier. If !cs still, use make_conn_state() to allocate it (this should not happen anymore though). * server/mpm/event/event.c(push2worker): The cs (if any) is now created by the caller, remove the csd and ptrans arguments. * server/mpm/event/event.c(push2pending): New helper to push a cs to the pending_q after disabling the listening sockets. * server/mpm/event/event.c(get_worker): No need to block anymore thanks to the pending_q, so remove the "blocking" argument and ap_queue_info_wait_for_idler() call. * server/mpm/event/event.c(listener_thread): If get_worker() returns no idle worker, push the PT_CSD and PT_ACCEPT connections to the pending queue (using the push2pending() helper). For the PT_CSD case there is no need for the "blocking" variable anymore since get_worker() is always non-blocking. For the PT_ACCEPT case the cs is now created before push2worker() or push2pending() to account for the connection ASAP, otherwise a graceful restart happening before it's processed would consider it does not exist and could exit the child too early (e.g. test_h2_004_22 from the pytest suites exercises this). The pending_q is processed for timeouts with the others (step 6). * server/mpm/event/event.c(worker_thread): Process the entries in the pending_q before returning to idle state. * server/mpm/event/event.c(event_post_config): Create the pending_q.

Now that the listener loop is fully non-blocking, we can ask for and use only two timestamps accross loop: one for before poll()ing and the other one for after poll()ing. After poll()ing the timestamp might drift a bit while processing all the events, but if the queues maintenance is skipped because of that it will happen in the next loop with timeout = 0. Also process_timeout_queue() can be quite simplified if it does not need to release the lock (all the callbacks are non-blocking too). * server/mpm/event/event.c(TIMEOUT_EXPIRED): New helper to determine whether a timestamp + timeout expired, also expiring entries if the clock went backwart too much. * server/mpm/event/event.c(listener_thread): Use two time1 and time2 variables for the time(stamp)s before and after poll()ing. time2 is fetched just after poll() and reused for all the below processing (i.e. not recomputed for queues maintenance). * server/mpm/event/event.c(process_timeout_queue): Simplify the code by calling the callback in a single (double-)loop. The timeout_mutex mutex is not released/re-acquired anymore. Use TIMEOUT_EXPIRED() to check for expiry.

Regardless of keep_alive_timeout_set which anyway is only about the KeepAliveTimeout to apply _after_ the current request, always use the request's server Timeout during its processing (i.e. READ_REQUEST_LINE and WRITE_COMPLETION). To save the next KeepAliveTimeout to use later, add a new event_srv_cfg to the conn_state which points to the appropriate server (either r->server or c->base_server depending on keep_alive_timeout_set as before). * server/mpm/event/event.c(struct event_conn_state_t): Add event_srv_cfg *ka_sc as the server config to apply for kept alive connections. * server/mpm/event/event.c(event_post_read_request): Always set cs->sc to the event_srv_cfg or the request's server, and point cs->ka_sc to the appropriate one according to keep_alive_timeout_set. * server/mpm/event/event.c(make_conn_state): Initialize cs->ka_sc to the ap_server_conf's event_srv_cfg, like cs->sc. * server/mpm/event/event.c(process_socket): Use cs->ka_sc->ka_q for CONN_STATE_CHECK_REQUEST_LINE_READABLE.

If clock_gettime() and CLOCK_MONOTONIC are defined (i.e. most/all? unixes), use them to provide a timestamp that never goes past (even if the admin changes the system time). This avoids entries potentially suddenly expiring in centuries on a bad clock skew. * configure.in(): Provide HAVE_TIME_H and HAVE_CLOCK_GETTIME. * server/mpm/event/event.c(event_time_now): New helper to get a monotonic timestamp from clock_gettime() if it's available, or apr_time_now() (i.e. gettimeofday()) otherwise. * server/mpm/event/event.c(TIMEOUT_EXPIRED): No need to check for a backward clock if it's monotonic. * server/mpm/event/event.c(process_socket, event_resume_suspended, event_get_timer_event, process_lingering_close, listener_thread, event_run): Use event_time_now().

* server/mpm/event/event.c(TO_QUEUE_CHAIN): New helper to do the chaining. * server/mpm/event/event.c(event_post_config): Use TO_QUEUE_CHAIN() to create the {rl,wc,ka}_q per server and axe a lot of duplicated code.

I must have been high when I wrote this comment.. * server/mpm/event/event.c(timer_comp): Simpler comment, code with no casts.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Event async #290

Event async #290

Commits on Feb 3, 2022