Skip to content
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Shrink the shared JavaScript chunk by about 88 KB (11 KB gzipped) when `frontend_lazy_bundled_libraries` is on: server-provided scripts and callbacks no longer use a direct `eval`, so the minifier can shorten every name in that chunk. Scripts passed to `rx.call_script` and `rx.call_function` now see only `addEvents`, `ReflexEvent`, `applyEventActions` and the globals, not other internals of the runtime module.
Comment thread
FarhanAliRaza marked this conversation as resolved.
Resolved
Comment thread
FarhanAliRaza marked this conversation as resolved.
Resolved
4 changes: 2 additions & 2 deletions packages/reflex-base/src/reflex_base/utils/format.py
Original file line number Diff line number Diff line change
Expand Up @@ -523,8 +523,8 @@ def format_queue_events(

The resulting code can be passed to interfaces that expect a callback
function and when triggered it dispatches the events through `addEvents`.
The name resolves via the attached imports in compiled modules and via a
local alias in the `applyResultCallback` eval context.
The name resolves via the attached imports in compiled modules and via an
explicit environment in the `applyResultCallback` execution context.

Args:
events: The events to queue.
Expand Down
14 changes: 8 additions & 6 deletions packages/reflex-frontend/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,12 +57,14 @@ without it appearing in its arguments. What there is, and where it is pinned:
evaluates every module in node to prerender pages, so anything added here has
to load without either. `tests/public_api.test.js` imports each module in
node to keep it that way.
- **`eval` scope**: event code sent by the backend (`rx.call_script`, event
callbacks) is evaluated inside `utils/state.js` and resolves names from the
scopes around the `eval`: `queueEvents`, `processEvent`, `refs`, `addEvents`,
and the `socket`, `navigate` and `params` parameters. This is why the file is
one module, why the linter's unused-variable rules are off for it
(`lint.biome.jsonc`), and why nothing there is renamed or moved casually.
- **Script scope**: event code sent by the backend (`rx.call_script`, event
callbacks) runs through `runScript` in `utils/state.js`, which compiles it
with the Function constructor and passes in every export of that module, the
late-bound `addEvents`, the `socket`, `navigate` and `params` of the event
being applied, and the session `token`. A direct `eval` would see every
module local, but it stops the minifier from mangling the chunk that holds
it. So a script sees those names and the globals, and nothing else. Renaming
an export changes what compiled callbacks can call.
- **Globals the runtime reads**: hooks other code installs on `window`
(`__reflex_otel`, `__reflex`, `__reflex_load`) and the per-app `$/env.json`.
Each is declared in `types/globals.d.ts`; reading one that is not fails the
Expand Down
85 changes: 69 additions & 16 deletions packages/reflex-frontend/src/utils/state.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ import {
useParams,
} from "react-router";
import { app, eventLoop } from "$/utils/context-registry";
// This module's own exports, passed to server-provided scripts by runScript.
import * as stateExports from "$/utils/state";
import {
CLIENT_STATE_REF,
getClientState,
Expand Down Expand Up @@ -469,7 +471,7 @@ export const evalReactComponent = async (component) => {
}
const encodedJs = encodeURIComponent(component);
const dataUri = "data:text/javascript;charset=utf-8," + encodedJs;
const module = await eval(`import(dataUri)`);
const module = await import(/* @vite-ignore */ dataUri);
return module.default;
};

Expand Down Expand Up @@ -508,14 +510,57 @@ function urlFrom(string) {
}
}

/**
* Run server-provided JavaScript with this module's exports in scope.
*
* Callback strings and `rx.call_script` code reference helpers compiled
* pages import from here (`ReflexEvent`, `pyMod`, `isTrue`, ...), and apps
* written against the former direct eval also use `socket`, `navigate`,
* `params` and `token`. A direct eval would see them, but it also stops the
* minifier from mangling every name in the chunk that holds it, so the code
* is compiled with the Function constructor and those names are passed in as
* parameters. The eval inside the body still runs the code as a strict
* script, so statements keep their completion value.
*
* Compiled on first use: a Content Security Policy without `unsafe-eval`
* rejects the Function constructor, and apps that never send scripts must
* still load.
*
* @param code The JavaScript source to run.
* @param addEvents The late-bound event dispatcher for this call.
* @param socket The socket object to send events on.
* @param navigate The navigate function from useNavigate.
* @param params The params object from useParams.
* @returns The completion value of the script.
*/
let evalInScope;
const runScript = (code, addEvents, socket, navigate, params) => {
evalInScope ??= new Function(
...Object.keys(stateExports),
"addEvents",
"socket",
"navigate",
"params",
"token",
"code",
'"use strict"; return eval(code);',
);
return evalInScope(
...Object.values(stateExports),
addEvents,
socket,
navigate,
params,
token,
code,
);
};

/**
* Invoke an event's result callback, if it declared one.
*
* The callback arrives as a string built by ``format_queue_events``, which
* references ``queueEvents``/``processEvent`` (module-level here) plus ``socket``,
* ``navigate`` and ``params``. Those three MUST stay the parameter names below:
* the ``eval`` resolves them from this function's scope, so renaming them breaks
* every callback.
* references ``addEvents`` and ``ReflexEvent``; ``runScript`` provides them.
* @param event The event whose callback to run.
* @param eval_result The value to pass to the callback, awaited if thenable.
* @param socket The socket object to send events on.
Expand All @@ -532,18 +577,16 @@ const applyResultCallback = async (
if (!event.payload.callback) {
return;
}
// Eval'd user code dispatches through addEvents like compiled event triggers
// do; late-bound so a remounted EventLoopProvider is picked up. Declared in
// every scope holding an eval, since that is the only scope chain the eval'd
// code can resolve it from.
// User code dispatches through addEvents like compiled event triggers do;
// late-bound so a remounted EventLoopProvider is picked up.
const addEvents = (...args) => eventLoop.addEvents(...args);
const final_result =
!!eval_result && typeof eval_result.then === "function"
? await eval_result
: eval_result;
const callback =
typeof event.payload.callback === "string"
? eval(event.payload.callback)
? runScript(event.payload.callback, addEvents, socket, navigate, params)
: event.payload.callback;
callback(final_result);
};
Expand Down Expand Up @@ -588,10 +631,8 @@ const storageDefault = (state_key) => {
* @param params The params object from useParams
*/
export const applyEvent = async (event, socket, navigate, params) => {
// Eval'd user code dispatches through addEvents like compiled event triggers
// do; late-bound so a remounted EventLoopProvider is picked up. Declared in
// every scope holding an eval, since that is the only scope chain the eval'd
// code can resolve it from.
// User code dispatches through addEvents like compiled event triggers do;
// late-bound so a remounted EventLoopProvider is picked up.
const addEvents = (...args) => eventLoop.addEvents(...args);
// Handle special events
if (event.name === "_redirect") {
Expand Down Expand Up @@ -786,8 +827,20 @@ export const applyEvent = async (event, socket, navigate, params) => {
try {
const eval_result =
event.name === "_call_script"
? eval(event.payload.javascript_code)
: eval(event.payload.function)();
? runScript(
event.payload.javascript_code,
addEvents,
socket,
navigate,
params,
)
: runScript(
event.payload.function,
addEvents,
socket,
navigate,
params,
)();

await applyResultCallback(event, eval_result, socket, navigate, params);
} catch (e) {
Expand Down
122 changes: 122 additions & 0 deletions packages/reflex-frontend/tests/run_script.test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
/**
* Server-provided JavaScript (`_call_script`, `_call_function` and their
* callbacks) runs through a Function-constructor helper, not a direct eval, so
* the minifier may mangle this module. These pin what the generated strings
* can see and what the script returns.
*/
import { describe, expect, test, vi } from "vitest";

import { eventLoop } from "$/utils/context-registry";
import { applyEvent, getToken } from "$/utils/state";

const callScript = (code, callback, socket = null) =>
applyEvent(
{ name: "_call_script", payload: { javascript_code: code, callback } },
socket,
null,
null,
);

describe("_call_script", () => {
test("returns the completion value of a statement list to the callback", async () => {
const seen = [];
await callScript("var x = 20; x + 1", (result) => seen.push(result));
expect(seen).toEqual([21]);
});

test("callback strings dispatch through the late-bound addEvents", async () => {
eventLoop.addEvents = vi.fn();
await callScript(
"42",
'((_result) => addEvents([ReflexEvent("state.set", {v: _result})], [_result], {}))',
);
expect(eventLoop.addEvents).toHaveBeenCalledTimes(1);
const [events, args] = eventLoop.addEvents.mock.calls[0];
expect(events[0].name).toBe("state.set");
expect(events[0].payload).toEqual({ v: 42 });
expect(args).toEqual([42]);
});

test("runs strict and sees no other module-local names", async () => {
const onerror = vi.fn();
window.onerror = onerror;
await callScript("undeclared = 1");
await callScript("eventLoop");
expect(onerror).toHaveBeenCalledTimes(2);
expect(onerror.mock.calls[0][4]).toBeInstanceOf(ReferenceError);
expect(onerror.mock.calls[1][4]).toBeInstanceOf(ReferenceError);
});

test("scripts keep the socket, navigate, params and token names", async () => {
// Apps written against direct eval call socket.disconnect() and read the
// session token from a script.
const socket = { disconnect: vi.fn() };
const token = getToken();
expect(typeof token).toBe("string");
const seen = [];
await callScript(
"socket.disconnect(); [typeof navigate, typeof params, token]",
(result) => seen.push(result),
socket,
);
expect(socket.disconnect).toHaveBeenCalledTimes(1);
expect(seen).toEqual([["object", "object", token]]);
});

test("callback strings can use every helper this module exports", async () => {
// Var operations inside a callback compile to helpers such as pyMod,
// isTrue or pyStrLength, which compiled pages import but a string cannot.
eventLoop.addEvents = vi.fn();
await callScript(
"7",
'((_result) => addEvents([ReflexEvent("s.set", {x: pyMod(_result, 2), y: isTrue(_result), z: pyStrLength("ab")})]))',
);
expect(eventLoop.addEvents).toHaveBeenCalledTimes(1);
expect(eventLoop.addEvents.mock.calls[0][0][0].payload).toEqual({
x: 1,
y: true,
z: 2,
});
});

test("callbacks dispatch through the event loop mounted at call time", async () => {
const first = vi.fn();
const second = vi.fn();
const event = { payload: { callback: "(value) => addEvents([value])" } };
eventLoop.addEvents = first;
await callScript("1", event.payload.callback);
eventLoop.addEvents = second;
await callScript("2", event.payload.callback);
expect(first).toHaveBeenCalledWith([1]);
expect(second).toHaveBeenCalledWith([2]);
});

test("the evaluator is compiled on first use, not on import", async () => {
// A Content Security Policy without unsafe-eval rejects the Function
// constructor; apps that never send scripts must still load.
vi.resetModules();
const RealFunction = globalThis.Function;
globalThis.Function = function () {
throw new EvalError("blocked by CSP");
};
try {
await expect(import("$/utils/state")).resolves.toBeTruthy();
} finally {
globalThis.Function = RealFunction;
}
});

test("_call_function evaluates a string and calls it", async () => {
const seen = [];
await applyEvent(
{
name: "_call_function",
payload: { function: "() => 7", callback: (r) => seen.push(r) },
},
null,
null,
null,
);
expect(seen).toEqual([7]);
});
});
2 changes: 2 additions & 0 deletions tests/units/compiler/state_js.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -239,6 +239,8 @@ async function setup({
},
});
await module.link((name) => {
// state.js imports itself to hand its exports to server-provided scripts.
if (name === "$/utils/state") return module;
assert.ok(imports[name], `unexpected import: ${name}`);
return new vm.SyntheticModule(
Object.keys(imports[name]),
Expand Down
68 changes: 14 additions & 54 deletions tests/units/compiler/test_state_js_template.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,7 @@

ADD_EVENTS_ALIAS = "const addEvents = (...args) => eventLoop.addEvents(...args);"

USER_CODE_EVALS = (
"eval(event.payload.callback)",
"eval(event.payload.javascript_code)",
"eval(event.payload.function)",
)
USER_CODE_RUNS = re.compile(r"runScript\(\s*event\.payload\.(\w+),\s*addEvents\b")
Comment thread
FarhanAliRaza marked this conversation as resolved.
Resolved

TOP_LEVEL_FUNCTION = re.compile(
r"^(?:export )?const (\w+) = (?:async )?\(", re.MULTILINE
Expand All @@ -41,42 +37,6 @@ def _enclosing_function(content: str, index: int) -> tuple[str, str]:
]


def test_result_callback_dispatches_through_current_event_loop() -> None:
"""Serialized callbacks can dispatch after extraction into their own helper."""
node = shutil.which("node")
if node is None:
pytest.skip("Node.js is needed to execute the callback")
content = STATE_JS_TEMPLATE.read_text()
helper = content[
content.index("const applyResultCallback =") : content.index(
"export const applyEvent ="
)
]
subprocess.run(
[
node,
"--input-type=module",
"--eval",
"""
import assert from 'node:assert/strict';
const calls = [];
const eventLoop = {addEvents: (events) => calls.push(events)};
"""
+ helper
+ """
const event = {payload: {callback: '(value) => addEvents([value])'}};
await applyResultCallback(event, Promise.resolve('first'), null, null, null);
eventLoop.addEvents = (events) => calls.push(['remounted', ...events]);
await applyResultCallback(event, 'second', null, null, null);
assert.deepEqual(calls, [['first'], ['remounted', 'second']]);
""",
],
check=True,
capture_output=True,
text=True,
)


def test_socket_startup_lifecycle() -> None:
"""Execute the frontend startup and reconnect tests against the real template."""
node = shutil.which("node")
Expand Down Expand Up @@ -133,25 +93,25 @@ def test_state_js_still_handles_page_lifecycle_disconnect() -> None:


def test_every_user_code_eval_can_reach_add_events() -> None:
"""Every scope that evals user code must declare the `addEvents` alias.
"""Every scope that runs user code must declare the `addEvents` alias.

A direct `eval` resolves names from its own scope chain, so the alias has to
be declared in each function holding one. Moving it, or extracting an `eval`
into a new helper without it, leaves the eval'd code -- serialized callbacks,
`_call_script`, and the handlers `_call_function` returns -- referencing an
undefined `addEvents`.
`runScript` takes the late-bound alias as a parameter, so each function
passing user code -- serialized callbacks, `_call_script`, and the handlers
`_call_function` returns -- has to declare it first. The dispatch itself is
covered by `packages/reflex-frontend/tests/run_script.test.js`.
"""
content = STATE_JS_TEMPLATE.read_text()

for snippet in USER_CODE_EVALS:
eval_at = content.index(snippet)
name, body = _enclosing_function(content, eval_at)
sites = list(USER_CODE_RUNS.finditer(content))
assert [m.group(1) for m in sites] == ["callback", "javascript_code", "function"]
for site in sites:
name, body = _enclosing_function(content, site.start())
assert ADD_EVENTS_ALIAS in body, (
f"`{name}` evals user code via `{snippet}` but does not declare the "
"addEvents alias, so the eval'd code cannot resolve it."
f"`{name}` runs `{site.group()}` but does not declare the "
"addEvents alias it passes."
)
assert body.index(ADD_EVENTS_ALIAS) < body.index(snippet), (
f"`{name}` declares the addEvents alias after `{snippet}`."
assert body.index(ADD_EVENTS_ALIAS) < body.index(site.group()), (
f"`{name}` declares the addEvents alias after `{site.group()}`."
)


Expand Down
Loading
Loading