Skip to content

Commit

Permalink
Adds v0.4.0 code
Browse files Browse the repository at this point in the history
  • Loading branch information
msmathers committed Jan 10, 2020
1 parent a004ce6 commit bd65317
Show file tree
Hide file tree
Showing 58 changed files with 27,722 additions and 0 deletions.
24 changes: 24 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.

# dependencies
node_modules/
/.pnp
.pnp.js

# testing
/coverage

# production
build/
lib/

# misc
.DS_Store
.env.local
.env.development.local
.env.test.local
.env.production.local

npm-debug.log*
yarn-debug.log*
yarn-error.log*
2 changes: 2 additions & 0 deletions .npmignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# Original source code
src/
13 changes: 13 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
Copyright 2020 Narrative Science

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
152 changes: 152 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
Log.io - Real-time log monitoring in your browser
=================================================

Powered by [node.js](http://nodejs.org) + [socket.io](http://socket.io)

## How does it work?

A **file input** watches log files for changes, sends new messages to the **server** via TCP, which broadcasts to **browsers** via socket.io.

Users can watch adhoc log streams by activating inputs and binding them to multiple screens via the web UI.

## Terminology

**Stream** - A logical designation for a group of messages that relate to one another. Examples include an application name, a topic name, or a backend service name.

**Source** - A physical designation for a group of messages that originate from the same source. Examples include a server name, a service provider name, or a filename.

**Input** - A (stream, source) pair.

While originally designed to represent backend service logs spread across multiple servers, the stream/source abstraction is intentionally open-ended to allow users to define a system topology for their specific use case.

## Install & run server

1) Install via npm

```
npm install log.io
```

2) Configure hosts & ports (see example below)

```
nano ~/.log.io/server.json
```

3) Run server

```
log.io-server
```

3) Browse to http://localhost:6688

## Install & run file input

1) Install via npm

```
npm install log.io-file-input
```

2) Configure file input (see example below)

```
nano ~/.log.io/inputs/file.json
```

3) Run file input

```
log.io-file-input
```

## Server configuration

There are two servers: the message server, which receives TCP messages from message inputs, and the HTTP server, which receives requests from browsers. By default, the application looks for configuration in `~/.log.io/server.json`, and can be overridden with the environment variable `LOGIO_SERVER_CONFIG_PATH`.

Sample configuration file:

```json
{
"messageServer": {
"port": 6689,
"host": "127.0.0.1"
},
"httpServer": {
"port": 6688,
"host": "127.0.0.1"
},
"debug": false
}

```

## File input configuration

Inputs are created by associating file paths with stream and source names in a configuration file. By default, the file input looks for configuration in `~/.log.io/inputs/file.json`, and can be overridden with the environment variable `LOGIO_FILE_INPUT_CONFIG_PATH`.

Sample configuration file:

```json
{
"messageServer": {
"host": "127.0.0.1",
"port": 6689
},
"inputs": [
{
"source": "server1",
"stream": "app1",
"config": {
"path": "log.io-demo/file-generator/app1-server1.log"
}
},
{
"source": "server2",
"stream": "app1",
"config": {
"path": "log.io-demo/file-generator/app1-server2.log"
}
},
{
"source": "server1",
"stream": "app2",
"config": {
"path": "log.io-demo/file-generator/app2-server1.log"
}
},
{
"source": "server2",
"stream": "app2",
"config": {
"path": "log.io-demo/file-generator/app2-server2.log"
}
}
]
}

```


## Server TCP interface

The file input connects to the server via TCP, and writes properly formatted strings to the socket. Custom inputs can send messages to the server using the following commands, each of which ends with a null character:

Send a log message

```
+msg|streamName1|sourceName1|this is log message\0
```

Register a new input

```
+input|streamName1|sourceName1\0
```

Remove an existing input

```
-input|streamName1|sourceName1\0
```
10 changes: 10 additions & 0 deletions inputs/file/.eslintrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"parser": "@typescript-eslint/parser",
"plugins": ["@typescript-eslint"],
"extends": ["airbnb-base", "plugin:@typescript-eslint/recommended"],
"rules": {
"import/no-unresolved": "off",
"semi": "off",
"@typescript-eslint/member-delimiter-style": "off"
}
}
11 changes: 11 additions & 0 deletions inputs/file/babel.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
module.exports = {
presets: [
"@babel/preset-env",
"@babel/typescript"
],
plugins: [
"@babel/proposal-class-properties",
"@babel/proposal-object-rest-spread",
"@babel/transform-runtime",
]
}
2 changes: 2 additions & 0 deletions inputs/file/bin/log.io-file-input
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/usr/bin/env node
require("../lib/index.js")
36 changes: 36 additions & 0 deletions inputs/file/config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
{
"messageServer": {
"host": "127.0.0.1",
"port": 6689
},
"inputs": [
{
"source": "server1",
"stream": "app1",
"config": {
"path": "log.io-demo/file-generator/app1-server1.log"
}
},
{
"source": "server2",
"stream": "app1",
"config": {
"path": "log.io-demo/file-generator/app1-server2.log"
}
},
{
"source": "server1",
"stream": "app2",
"config": {
"path": "log.io-demo/file-generator/app2-server1.log"
}
},
{
"source": "server2",
"stream": "app2",
"config": {
"path": "log.io-demo/file-generator/app2-server2.log"
}
}
]
}
Loading

0 comments on commit bd65317

Please sign in to comment.