Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Spice.ai API Key
# Get your API key from https://spice.ai
SPICE_API_KEY=your_api_key_here

# Optional: Override default endpoints for testing
# HTTP_URL=https://data.spiceai.io
# FLIGHT_URL=flight.spiceai.io:443

# Optional: Vercel deployment testing
# VERCEL_ENDPOINT=https://spice-js.vercel.app/api
# VERCEL_AUTOMATION_BYPASS_SECRET=your_secret_here
15 changes: 4 additions & 11 deletions .github/workflows/npm-publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,18 +32,11 @@ jobs:
vercel-bypass-secret: ${{ secrets.VERCEL_AUTOMATION_BYPASS_SECRET }}
branch-name: ${{ github.ref_name }}

- run: |
npm run test -t 'cloud'
- name: Test
run: npm test
timeout-minutes: 10
env:
SPICEAI_API_KEY: ${{ secrets.SPICEAI_API_KEY }}
API_KEY: ${{ secrets.API_KEY }}
RELAY_KEY: ${{ secrets.RELAY_KEY }}
RELAY_SECRET: ${{ secrets.RELAY_SECRET }}
RELAY_URL: ${{ secrets.RELAY_URL }}
HTTP_URL: ${{ secrets.HTTP_URL || 'https://data.spiceai.io' }}
FLIGHT_URL: ${{ secrets.FLIGHT_URL || 'flight.spiceai.io:443' }}
VERCEL_ENDPOINT: ${{ steps.vercel.outputs.vercel-api-endpoint }}
VERCEL_AUTOMATION_BYPASS_SECRET: ${{ secrets.VERCEL_AUTOMATION_BYPASS_SECRET }}
SPICE_API_KEY: ${{ secrets.SPICE_API_KEY }}

publish-npm:
needs: build
Expand Down
5 changes: 3 additions & 2 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -184,9 +184,10 @@ jobs:
run: npm run build

- name: Run cloud tests
run: npm run test:node -- test/cloud.test.ts
run: npm test -- test/cloud.test.ts
timeout-minutes: 10
env:
SPICEAI_API_KEY: ${{ secrets.SPICEAI_API_KEY }} # spice.ai/spiceai/spice-js
SPICE_API_KEY: ${{ secrets.SPICE_API_KEY }} # spice.ai/spiceai/spice-js
HTTP_URL: ${{ secrets.HTTP_URL || 'https://data.spiceai.io' }}
FLIGHT_URL: ${{ secrets.FLIGHT_URL || 'flight.spiceai.io:443' }}
VERCEL_ENDPOINT: ${{ needs.vercel-setup.outputs.vercel-api-endpoint }}
Expand Down
12 changes: 11 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,18 @@ To develop locally:

4. To run the tests, create a `.env` file with your [Spice.ai](https://spice.ai) API Key:

```bash
# Copy the example file
cp .env.example .env

# Edit .env and add your API key
# SPICE_API_KEY=your_api_key_here
```

Or set the environment variable directly:

```env
API_KEY=<Spice.ai API Key>
SPICE_API_KEY=<Spice.ai API Key>
```

5. Run the tests with:
Expand Down
90 changes: 85 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,14 +54,33 @@ const main = async () => {
// });

const table = await spiceClient.sql(
'SELECT trip_distance, total_amount FROM taxi_trips ORDER BY trip_distance DESC LIMIT 10;',
'SELECT trip_distance, total_amount FROM taxi_trips ORDER BY trip_distance DESC LIMIT 10;'
);
console.table(table.toArray());
};

main();
```

### Automatic Transport Selection

The SpiceClient automatically selects the best available transport protocol in this order:

1. **Arrow Flight SQL** - gRPC protocol with parameter substitution
2. **HTTP/HTTPS** - Fallback for browser environments or when Flight is unavailable

For parameterized queries, the SDK provides secure parameter binding:

```js
// Parameterized query using Flight SQL or HTTP
const table = await client.sql(
'SELECT * FROM taxi_trips WHERE passenger_count = $1 AND trip_distance > $2 LIMIT 10',
{ parameters: [2, 5.0] }
);
```

The SDK handles all protocol negotiation automatically - you just write standard SQL with parameters.

## Upgrading from v2 to v3

Version 3.0 represents a major evolution of the SDK with cross-platform support, new APIs, and enhanced reliability.
Expand Down Expand Up @@ -377,7 +396,7 @@ pnpm add @spiceai/spice@latest

## API Methods

### `sql(query: string, onData?: callback)` - Execute SQL queries
### `sql(query: string, options?: SqlQueryOptions, onData?: callback)` - Execute SQL queries

The `sql()` method executes SQL queries and returns results as Apache Arrow tables. This is the recommended method for querying data.

Expand Down Expand Up @@ -489,7 +508,7 @@ The `nsql()` method converts natural language queries into SQL and executes them
```js
// Basic natural language query
const result = await spiceClient.nsql(
'Show me the top 5 customers by total sales',
'Show me the top 5 customers by total sales'
);

console.log('Generated SQL:', result.sql);
Expand All @@ -503,7 +522,7 @@ const result = await spiceClient.nsql(
datasets: ['taxi_trips'], // Limit to specific datasets
model: 'nql', // Specify the model (default: 'nql')
sample_data_enabled: true, // Include sample data in context (default: true)
},
}
);

// Access the generated SQL
Expand Down Expand Up @@ -561,6 +580,47 @@ The `SpiceClient` automatically handles environments where Apache Arrow Flight g

Both gRPC and HTTP modes support compression (gzip, deflate) to reduce bandwidth usage. This ensures the SDK works efficiently in any environment without configuration changes. See [docs/http-fallback.md](./docs/http-fallback.md) for more details.

## Advanced

### Parameterized Queries

The SpiceClient automatically supports parameterized queries through its `.sql()` method. Parameters are handled transparently using the best available protocol (Flight SQL → HTTP).

**Basic usage:**

```js
import { SpiceClient } from '@spiceai/spice';

const client = new SpiceClient({
apiKey: 'YOUR_API_KEY',
httpUrl: 'https://data.spiceai.io',
flightUrl: 'flight.spiceai.io:443',
});

// Positional parameters (using $1, $2, etc.)
const table = await client.sql(
'SELECT * FROM taxi_trips WHERE trip_distance > $1 AND passenger_count >= $2 LIMIT 10',
{ parameters: [5.0, 2] }
);

console.table(table.toArray());
```

**Transport Hierarchy:**

When parameters are provided, the SDK automatically:

1. **Uses Flight SQL** - Client-side parameter substitution with Arrow Flight
2. **Falls back to HTTP** - Sends parameters as JSON if Flight is unavailable

**Key benefits:**

- **SQL Injection Prevention**: Parameters are properly escaped and validated
- **Type Safety**: Parameters maintain their data types
- **Automatic fallback**: Works in all environments (Node.js and browser)

For more information, see [docs/PARAMETERIZED_QUERIES.md](./docs/PARAMETERIZED_QUERIES.md).

## Documentation

Check out our [API documentation](https://docs.spice.ai/sdks/node.js-sdk) to learn more about how to use the Node.js SDK.
Expand All @@ -577,6 +637,26 @@ npm run test:perf

For more details, see [docs/PERFORMANCE_TESTING.md](./docs/PERFORMANCE_TESTING.md).

## Running tests locally
## Development

### Environment Setup

For development and testing, you'll need to set up environment variables:

1. Copy the example environment file:

```bash
cp .env.example .env
```

2. Edit `.env` and add your [Spice.ai](https://spice.ai) API key:

```env
SPICE_API_KEY=your_api_key_here
```

The `.env` file is automatically loaded by the test suite and can be used by examples.

### Running Tests Locally

Run the tests with `make test`. For more information, see [CONTRIBUTING.md](./CONTRIBUTING.md)
4 changes: 2 additions & 2 deletions docs/CI_TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,7 @@ The CI uses the same commands as local development:

### Required for Cloud Tests

- `SPICEAI_API_KEY` - Spice.ai API key (from GitHub Secrets)
- `SPICE_API_KEY` - Spice.ai API key (from GitHub Secrets)

### Optional

Expand Down Expand Up @@ -308,7 +308,7 @@ npm run test:browser
npm run test:node

# Cloud tests (requires API key)
SPICEAI_API_KEY=your_key npm run test -- test/cloud.test.ts
SPICE_API_KEY=your_key npm run test -- test/cloud.test.ts
```

## Performance
Expand Down
152 changes: 152 additions & 0 deletions examples/parameterized-queries.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
/**
* Example demonstrating parameterized queries with Apache Flight SQL
*
* The SpiceClient automatically uses Apache Flight SQL (via gRPC) for parameterized
* queries with client-side parameter substitution. This provides security against
* SQL injection while maintaining compatibility with all Flight SQL implementations.
*/

const { SpiceClient } = require('@spiceai/spice');

async function main() {
console.log('='.repeat(80));
console.log('Parameterized Queries with Apache Flight SQL');
console.log('='.repeat(80));

// Initialize client
const client = new SpiceClient({
apiKey: process.env.SPICE_API_KEY,
httpUrl: 'https://data.spiceai.io',
flightUrl: 'flight.spiceai.io:443',
});

try {
// Example 1: Positional parameters (recommended)
console.log('\n📊 Example 1: Positional Parameters ($1, $2, etc.)');
console.log('-'.repeat(80));

const table1 = await client.sql(
'SELECT * FROM taxi_trips WHERE passenger_count = $1 AND trip_distance > $2 LIMIT 10',
{ parameters: [2, 5.0] }
);

console.log(`✓ Query executed successfully`);
console.log(` Rows returned: ${table1.numRows}`);
console.log(
` Transport used: Apache Flight SQL (gRPC) with parameter substitution`
);
console.log('\nFirst few rows:');
console.table(table1.toArray().slice(0, 3));

// Example 2: Named parameters
console.log('\n📊 Example 2: Named Parameters ($param_name)');
console.log('-'.repeat(80));

const table2 = await client.sql(
'SELECT * FROM taxi_trips WHERE passenger_count = $passengers AND fare_amount > $min_fare LIMIT 10',
{
parameters: {
passengers: 3,
min_fare: 20.0,
},
}
);

console.log(`✓ Query executed successfully`);
console.log(` Rows returned: ${table2.numRows}`);
console.log('\nFirst few rows:');
console.table(table2.toArray().slice(0, 3));

// Example 3: Different data types
console.log('\n📊 Example 3: Multiple Data Types');
console.log('-'.repeat(80));

const table3 = await client.sql(
`SELECT * FROM taxi_trips
WHERE passenger_count >= $1
AND trip_distance BETWEEN $2 AND $3
AND store_and_fwd_flag = $4
LIMIT 10`,
{
parameters: [
2, // integer
1.0, // float (minimum distance)
10.0, // float (maximum distance)
'N', // string
],
}
);

console.log(`✓ Query executed successfully`);
console.log(` Rows returned: ${table3.numRows}`);
console.log('\nFirst few rows:');
console.table(table3.toArray().slice(0, 3));

// Example 4: Handling special characters (SQL injection prevention)
console.log('\n📊 Example 4: SQL Injection Prevention');
console.log('-'.repeat(80));

// This would be dangerous without proper escaping
const maliciousInput = "' OR '1'='1";

const table4 = await client.sql(
'SELECT COUNT(*) as count FROM taxi_trips WHERE store_and_fwd_flag = $1',
{ parameters: [maliciousInput] }
);

console.log(`✓ Query executed safely`);
console.log(` Input value: "${maliciousInput}"`);
console.log(` Parameters are properly escaped - no SQL injection!`);
console.log('\nResult:');
console.table(table4.toArray());

// Example 5: NULL handling
console.log('\n📊 Example 5: NULL Values');
console.log('-'.repeat(80));

const table5 = await client.sql(
'SELECT * FROM taxi_trips WHERE passenger_count = $1 OR $1 IS NULL LIMIT 5',
{ parameters: [null] }
);

console.log(`✓ NULL parameter handled correctly`);
console.log(` Rows returned: ${table5.numRows}`);

// Summary
console.log('\n' + '='.repeat(80));
console.log('✓ All examples completed successfully!');
console.log('='.repeat(80));
console.log('\n📝 Key Points:');
console.log(
' • Parameterized queries work perfectly with Apache Flight SQL'
);
console.log(' • Parameters are safely substituted on the client side');
console.log(
' • Both positional ($1, $2) and named ($param_name) parameters supported'
);
console.log(
' • Automatic SQL injection prevention through proper escaping'
);
console.log(
' • Works with all data types: strings, numbers, booleans, null'
);
console.log(' • Automatic fallback to HTTP if Flight SQL is unavailable');
console.log('\n💡 Best Practices:');
console.log(' • Use positional parameters for simple queries');
console.log(
' • Use named parameters for complex queries with many parameters'
);
console.log(' • Never concatenate user input directly into SQL strings');
console.log(' • Let the SDK handle parameter escaping automatically');
} catch (error) {
console.error('\n❌ Error:', error.message);
console.error(error.stack);
}
}

// Run the example
if (require.main === module) {
main().catch(console.error);
}

module.exports = { main };
3 changes: 2 additions & 1 deletion jest.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,5 @@
module.exports = {
preset: 'ts-jest',
testEnvironment: 'node',
};
setupFilesAfterEnv: ['<rootDir>/test/setup.ts'],
};
Loading
Loading