ui_elements is an experimental library for building stateful voice activated overlays and UIs using HTML/CSS/React-like syntax, for use with Talon.
- 9 Example UIs
- HTML-like elements such as
div,text,button,input_text - 40+ CSS-like properties such as
width,background_color,margin,padding_left,flex_direction - Reactive utilties
state,effect, andref - Talon actions for setting text, highlighting elements, and changing state
- Voice activated hints displayed on all buttons and text inputs
Download or clone this repository into your Talon user directory.
# mac and linux
cd ~/.talon/user
# windows
cd ~/AppData/Roaming/talon/user
git clone https://github.com/rokubop/talon-ui-elements.gitDone! 🎉 Say "elements test" to try out examples. Start learning below.
Choose elements from actions.user.ui_elements and create a renderer function in any .py file in your Talon user directory.
def hello_world_ui():
screen, div, text = actions.user.ui_elements(["screen", "div", "text"])
return screen()[
div()[
text("Hello world")
]
]To define styles, we put it inside of the parentheses. To define children, we put it inside the square brackets.
def hello_world_ui():
screen, div, text = actions.user.ui_elements(["screen", "div", "text"])
return screen(justify_content="center", align_items="center")[
div(background_color="333333", padding=16, border_radius=8, border_width=1)[
text("Hello world", font_size=24)
]
]Now we just need to show and hide it, so let's create two Talon actions. Here's the full .py code:
from talon import Module, actions
mod = Module()
def hello_world_ui():
screen, div, text = actions.user.ui_elements(["screen", "div", "text"])
return screen(justify_content="center", align_items="center")[
div(background_color="333333", padding=16, border_radius=8, border_width=1)[
text("Hello world", font_size=24)
]
]
@mod.action_class
class Actions:
def show_hello_world():
"""Show hello world UI"""
actions.user.ui_elements_show(hello_world_ui)
def hide_hello_world():
"""Hide hello world UI"""
actions.user.ui_elements_hide_all()
# or actions.user.ui_elements_hide(hello_world_ui)And in any .talon file:
show hello world: user.show_hello_world()
hide hello world: user.hide_hello_world()Now when you say "show hello world", the UI should appear.
Congratulations! You've created your first UI. 🎉
Start trying out properties to see how it changes. See all supported properties for styling.
Note: It's a good idea to say "Talon open log" and watch the log while developing. This will help you if you make any mistakes.
Checkout out examples in the examples folder. Or say "elements test" to view live interactive examples.
| Example | Preview | Description |
|---|---|---|
| alignment_ui | preview | Showcase 9 different flexbox arrangements |
| cheatsheet_ui | preview | A list of commands on the left or right of your screen that can change state |
| dashboard_ui | preview | Has a title bar, a side bar, and a reactive body |
| game_keys_ui | preview | Game keys overlay for gaming, that highlights respective keys |
| hello_world_ui | preview | Simple hello world UI |
| inputs_ui | preview | Text input, ref, validation, and submit with a button |
| state_vs_refs_ui | preview | Two versions of a counter using state or ref |
| icons_svgs_ui | preview | Icons and custom SVGs |
| todo_list_ui | preview | A todo list with an input, add, and remove functionality |
returned from actions.user.ui_elements:
| Element | Description |
|---|---|
screen |
Root element. A div the size of your screen. |
active_window |
Root element. A div the size of the currently active window. |
div |
Standard container element. |
text |
Basic strings supported. Combine multiple together if you want to style differently. |
button |
Accepts on_click |
icon |
See supported icons |
input_text |
Uses Talon's experimental TextArea for input. |
state |
Global reactive state that rerenders respective UIs when changed. |
effect |
Run side effects on mount, unmount, or state change. |
ref |
Reference to an element "id", which provides a way to imperatively get and set properties, with reactive updates. Useful for input_text value. |
ui_elements have the same box model as normal HTML, with margin, border, padding, and width and height and operate under box-sizing: border-box assumption, meaning border and padding are included in the width and height.
ui_elements are all display: flex, and default to flex_direction="column"with align_items="stretch". This means when you don't provide anything, it will act similarly to display: block.
If you aren't familiar with flexbox, check out this CSS Tricks Guide to Flexbox.
Some examples:
# children of screen will be bottom right
screen(align_items="flex_end", justify_content="flex_end")
# children of screen will be center
screen(align_items="center", justify_content="center")
# children of screen will be top left
screen(align_items="flex_start", justify_content="flex_start")
# children of screen will be top right
screen(flex_direction="row", align_items="flex_start", justify_content="flex_end")
# full width or height depending on flex_direction
div(flex=1)See alignment_ui for more.
..., state = actions.user.ui_elements([... "state"])
tab, set_tab = state.use("tab", 1)
# do conditional rendering with tabstate.use behaves like React’s useState. It returns a tuple (value, set_value). You must define a state key (e.g. "tab" in this case), so that actions.user.ui_elements* can also target it, and optionally a default value.
To change state, we can use set_tab from above, or we can use Talon actions:
actions.user.ui_elements_set_state("tab", 2)
actions.user.ui_elements_set_state("tab", lambda tab: tab + 1)State changes cause a full rerender (for now).
If the UI doesn't need a setter, than we can use state.get, which is just the value.
tab = state.get("tab", 1)Read more about state.
If you just need to update text or highlight, use the below methods instead, as those render on a separate decoration layer which are faster, and do not cause a full rerender.
We must give a unique id to the thing we want to update.
text("Hello world", id="test"),Then we can use this action to update the text:
actions.user.ui_elements_set_text("test", "New text")Simple text updates like this render on a separate decoration layer, and are faster than a full rerender.
We must give a unique id to the thing we want to update.
div(id="box", background_color="FF0000")[
text("Hello world"),
]Then we can use ui_elements_set_property to update the properties. Changes will cause a full rerender. (for now)
actions.user.ui_elements_set_property("box", "background_color", "red")
actions.user.ui_elements_set_property("box", "width", "400")
actions.user.ui_elements_set_property("box", {
"background_color": "red",
"width": "400"
})div(id="box")[
text("Hello world"),
]We can use these actions to trigger a highlight or unhighlight, targeting an element with the id "box". Highlights happen on a separate decoration layer, and are faster than a full rerender.
actions.user.ui_elements_highlight("box")
actions.user.ui_elements_highlight_briefly("box")
actions.user.ui_elements_unhighlight("box")To use a custom highlight color, we can use the following property:
div(id="box", highlight_color="FF0000")[
text("Hello world"),
]or we can specify the highlight color in the action:
actions.user.ui_elements_highlight_briefly("box", "FF0000aa")If you use a button, the UI will block the mouse instead of being pass through, and voice activated hints will automatially appear on the button.
# button
button("Click me", on_click=lambda e: print("clicked")),
button("Click me", on_click=actions.user.ui_elements_hide_all),See inputs_ui for example.
commands = [
"left",
"right",
"up",
"down"
]
div(gap=8)[
text("Commands", font_weight="bold"),
*[text(command) for command in commands]
],# 50% opacity
div(background_color="FF0000", opacity=0.5)[
text("Hello world")
]
# or we can use the last 2 digits of the color
div(background_color="FF000088")[
text("Hello world")
]The following elements are supported for SVGs. For the most part it matches the HTML SVG spec.
Based on a standard view_box="0 0 24 24". You can use size to resize, and stroke_width to change the stroke width.
returned from actions.user.ui_elements_svg
| Element | Description |
|---|---|
svg |
Wrapper for SVG elements. |
path |
Accepts d attribute. |
circle |
Accepts cx, cy, and r attributes. |
rect |
Accepts x, y, width, height, rx, and ry attributes. |
line |
Accepts x1, y1, x2, and y2 attributes. |
polyline |
Accepts points attribute. |
polygon |
Accepts points attribute. |
# screen 1
screen(1, align_items="flex_end", justify_content="center")[
div()[
text("Hello world")
]
]
# or
screen(screen=2, align_items="flex_end", justify_content="center")[
div()[
text("Hello world")
]
]To enable dragging, we can use the draggable property on the top most div.
screen()[
div(draggable=True)[
text("Drag me")
]
]By default the entire area is draggable. To limit the dragging handle to a specific element, we can use the drag_handle=True property on the element we want to use as the handle.
screen()[
div(draggable=True)[
div(drag_handle=True)[
text("Header")
]
div()[
# body content
]
]
]When the UI is interactive (either draggable, or has buttons or inputs), then focus outlines appear when you tab through the elements. To change the color and width of the focus outline, you can use the following properties:
div(focus_outline_color="FF0000", focus_outline_width=4)[
text("Hello world")
]Keyboard shortcuts become available if the UI is interactive.
| Key | Description |
|---|---|
Tab |
Move focus to the next element |
Shift + Tab |
Move focus to the previous element |
Down |
Move focus to the next element |
Up |
Move focus to the previous element |
Enter |
Trigger the focused element |
Space |
Trigger the focused element |
Esc |
Hide all UIs |
The following properties cascade down to children elements:
colorfont_sizefont_familyopacityhighlight_colorfocus_outline_colorfocus_outline_width
| Documentation | Description |
|---|---|
| Actions | Talon actions you can use (actions.user.ui_elements*) |
| Defaults | Default values for all properties |
| Properties | List of all properties you can use |
| Icons and SVGs | List of supported icons and how to use custom SVGs |
| Effect | Side effects on mount, unmount, or state change |
| State | Global reactive state that rerenders respective UIs when changed |
| Ref | Reference to an element "id", which provides a way to imperatively get and set properties, with reactive updates |
While developing, you might get into a state where the UI gets stuck on your screen and you need to restart Talon. For this reason, it's recommended to have a "talon restart" command.
In a .talon file:
^talon restart$: user.talon_restart()
Inside of a .py file:
import os
from talon import Module, actions, ui
mod = Module()
@mod.action_class
class Actions:
def talon_restart():
"""restart talon"""
# for windows only
talon_app = ui.apps(pid=os.getpid())[0]
os.startfile(talon_app.exe)
talon_app.quit()-
Sometimes the UI may not refresh after saving the file. Try hiding the UI, saving the file again, and showing again.
-
Recommend using "Andreas Talon" VSCode extension + its dependency pokey command server, so you can get autocomplete for talon user actions, and hover over hint documentation on things like
actions.user.ui_elements()oractions.user.ui_elements_show().
Uses Talon's Canvas and Skia canvas integration under the hood, along with Talon's experimental TextArea for input.
none