Skip to content

Commit 33a5e9f

Browse files
gsidhwani-nrgsidhwani-nr
authored
Merge pull request #10 from newrelic/Queue-ConnOptimization
QueueCapacity and connection tuning
2 parents ce99fca + a7c672f commit 33a5e9f

13 files changed

+676
-192
lines changed

‎README.md‎

Lines changed: 56 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ Add the library to your project using Maven Central:
2828
<dependency>
2929
<groupId>com.newrelic.labs</groupId>
3030
<artifactId>custom-log4j2-appender</artifactId>
31-
<version>1.0.5</version>
31+
<version>1.0.6</version>
3232
</dependency>
3333
```
3434

@@ -38,7 +38,8 @@ Or, if using a locally built JAR file:
3838
<dependency>
3939
<groupId>com.newrelic.labs</groupId>
4040
<artifactId>custom-log4j2-appender</artifactId>
41-
<version>1.0.5</version>
41+
<version>1.0.6</version>
42+
4243
<scope>system</scope>
4344
<systemPath>${project.basedir}/src/main/resources/custom-log4j2-appender.jar</systemPath>
4445
</dependency>
@@ -58,57 +59,92 @@ Replace `[your-api-key]` with the ingest key obtained from the New Relic platfor
5859

5960
```xml
6061
<?xml version="1.0" encoding="UTF-8"?>
61-
<Configuration status="DEBUG" name="cloudhub" packages="com.newrelic.labs">
62+
<Configuration status="INFO" name="cloudhub" packages="com.newrelic.labs">
6263
<Appenders>
6364
<NewRelicBatchingAppender name="NewRelicAppender"
6465
apiKey="YOUR_API_KEY"
6566
apiUrl="https://log-api.newrelic.com/log/v1"
6667
logType="muleLog"
6768
applicationName="your-application-name"
68-
batchSize="5000"
69+
batchSize="2000"
6970
maxMessageSize="1048576"
7071
flushInterval="120000"
7172
customFields="businessGroup=exampleGroup,environment=production"
7273
mergeCustomFields="true"
73-
maxRetries="5"
74+
maxRetries="3"
75+
connPoolSize="10"
76+
queueCapacity="2097152"
7477
timeout="15000">
7578
<PatternLayout pattern="[%d{MM-dd HH:mm:ss}] %-5p %c{1} [%t]: %m%n"/>
7679
</NewRelicBatchingAppender>
7780
</Appenders>
7881
<Loggers>
7982
<AsyncRoot level="INFO">
83+
<!-- Comment out the NewRelicAppender reference to disable it -->
8084
<AppenderRef ref="NewRelicAppender"/>
8185
</AsyncRoot>
8286
</Loggers>
8387
</Configuration>
8488
```
8589

90+
8691
### Parameters
8792

88-
| Parameter | Required? | Default Value | Description |
89-
|---------------------|-----------|---------------|-----------------------------------------------------------------------------|
90-
| name | Yes | | Name used to register Log4j Appender |
91-
| apiKey | Yes | | API key for authenticating with New Relic's logging service |
92-
| apiUrl | Yes | | URL for New Relic's log ingestion API |
93-
| logType | No | "muleLog" | Type of log being sent |
94-
| applicationName | Yes | | Name of the application generating the logs |
95-
| batchSize | No | 5000 | Maximum number of log entries to batch together before sending to New Relic |
96-
| maxMessageSize | No | 1048576 | Maximum size (in bytes) of the payload to be sent in a single HTTP request |
97-
| flushInterval | No | 120000 | Interval (in milliseconds) at which the log entries are flushed to New Relic|
98-
| customFields | No | | Add extra context to your logs with custom fields, represented as comma-separated name-value pairs.|
99-
| mergeCustomFields | No | false | (Default: false) All custom fields will be available as `custom.field1`, `custom.field2` else `field1` , `field2` will be available as the main attributes |
100-
| maxRetries | No | 3 | Maximum number of retry attempts for sending logs. If logs cannot be sent successfully within the specified retries, they will be discarded. |
101-
| timeout | No | 30000 | Connection timeout (in milliseconds) for HTTP requests to New Relic's logging service. Adjust based on network conditions and server response times. |
10293

94+
| Parameter | Required? | Default Value | Description |
95+
|---------------------|-----------|------------------------|-----------------------------------------------------------------------------|
96+
| name | Yes | | Name used to register Log4j Appender |
97+
| apiKey | Yes | | API key for authenticating with New Relic's logging service |
98+
| apiUrl | Yes | | URL for New Relic's log ingestion API |
99+
| logType | No | "muleLog" | Type of log being sent |
100+
| applicationName | Yes | | Name of the application generating the logs |
101+
| batchSize | No | 2000 | Maximum number of log entries to batch together before sending to New Relic |
102+
| maxMessageSize | No | 1048576 | Maximum size (in bytes) of the payload to be sent in a single HTTP request |
103+
| flushInterval | No | 120000 | Interval (in milliseconds) at which the log entries are flushed to New Relic|
104+
| customFields | No | | Add extra context to your logs with custom fields, represented as comma-separated name-value pairs.|
105+
| mergeCustomFields | No | "false" | Whether to merge custom fields into the main log attributes |
106+
| maxRetries | No | 3 | Maximum number of retry attempts for sending logs |
107+
| connPoolSize | No | 5 | Size of the connection pool for HTTP requests |
108+
| queueCapacity | No | 2097152 | Maximum capacity (in bytes) of the log queue |
109+
| timeout | No | 30000 | Connection timeout (in milliseconds) for HTTP requests |
110+
111+
---
112+
113+
## Log4j2 Layouts
114+
115+
### PatternLayout
116+
117+
- **Purpose**: Formats logs in a customizable text format.
118+
- **Example**:
119+
```xml
120+
<PatternLayout pattern="[%d{MM-dd HH:mm:ss}] %-5p %c{1} [%t]: %m%n"/>
121+
```
122+
- **Use Case**: Ideal for human-readable logs.
103123

124+
### JsonLayout
125+
126+
- **Purpose**: Formats logs as JSON objects for structured logging.
127+
- **Example**:
128+
```xml
129+
<JsonLayout compact="true" eventEol="true"/>
130+
```
131+
- **Use Case**: Suitable for integration with log management systems.
132+
133+
### Choosing Layout
134+
135+
- **PatternLayout**: For text-based logs.
136+
- **JsonLayout**: For structured, machine-readable logs.
137+
138+
---
104139

105140
## Custom Fields [ v1.0.1 + ]
106141
Custom fields provide a way to include additional custom data in your logs. They are represented as comma-separated name-value pairs. This feature allows you to add more context to your logs, making them more meaningful and easier to analyze.
107142

108143
## Configuring Custom Fields as Subfields of Custom Fields [v1.0.3+]
109144
Starting from version 1.0.3, a new configuration parameter `mergeCustomFields` has been added. By default, all custom fields will be available as subfields under the `custom` field (e.g., `custom.field1`, `custom.field2`). If `mergeCustomFields` is set to `true`, custom fields will be available as main attributes (e.g., `field1`, `field2`).
110145

111-
146+
## Configuring queueCapacity and connPoolSize [v1.0.6+]
147+
Starting from version 1.0.6, the queueCapacity and connPoolSize parameters are exposed to allow for fine-tuning of the appender's performance, especially under high load conditions. These parameters help manage the flow of log data and the efficiency of network connections.
112148

113149
### TLS 1.2 Requirement
114150

‎custom-log4j2-appender/build-jar.gradle‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ jar {
2020
'Implementation-Title': 'Custom Log4j2 Appender',
2121
'Implementation-Vendor': 'New Relic Labs',
2222
'Implementation-Vendor-Id': 'com.newrelic.labs',
23-
'Implementation-Version': '1.0.5'
23+
'Implementation-Version': '1.0.6'
2424
)
2525
}
2626
}
@@ -53,7 +53,7 @@ publishing {
5353

5454
groupId = 'com.newrelic.labs'
5555
artifactId = 'custom-log4j2-appender'
56-
version = '1.0.5'
56+
version = '1.0.6'
5757

5858
pom {
5959
name = 'Custom Log4j2 Appender'

‎custom-log4j2-appender/build-shadowJar.gradle‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ shadowJar {
2727
'Implementation-Title': 'Custom Log4j2 Appender',
2828
'Implementation-Vendor': 'New Relic Labs',
2929
'Implementation-Vendor-Id': 'com.newrelic.labs',
30-
'Implementation-Version': '1.0.5'
30+
'Implementation-Version': '1.0.6'
3131
)
3232
}
3333
}
@@ -55,7 +55,7 @@ publishing {
5555

5656
groupId = 'com.newrelic.labs'
5757
artifactId = 'custom-log4j2-appender'
58-
version = '1.0.5'
58+
version = '1.0.6'
5959

6060
pom {
6161
name = 'Custom Log4j2 Appender'

‎custom-log4j2-appender/build.gradle‎

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
plugins {
22
id 'java'
3+
id 'com.github.johnrengelman.shadow' version '6.1.0'
34
id 'maven-publish'
45
}
56

@@ -15,18 +16,22 @@ dependencies {
1516
}
1617

1718
jar {
19+
enabled = false // Disable the standard JAR task
20+
}
21+
22+
shadowJar {
23+
archiveClassifier.set('') // Remove the 'all' classifier
24+
mergeServiceFiles()
1825
manifest {
1926
attributes(
2027
'Implementation-Title': 'Custom Log4j2 Appender',
2128
'Implementation-Vendor': 'New Relic Labs',
2229
'Implementation-Vendor-Id': 'com.newrelic.labs',
23-
'Implementation-Version': '1.0.5'
30+
'Implementation-Version': '1.0.6'
2431
)
2532
}
2633
}
2734

28-
29-
3035
tasks.withType(JavaCompile) {
3136
options.encoding = 'UTF-8'
3237
}
@@ -41,19 +46,16 @@ task sourcesJar(type: Jar) {
4146
from sourceSets.main.allSource
4247
}
4348

44-
45-
4649
publishing {
4750
publications {
4851
mavenJava(MavenPublication) {
49-
from components.java // Include standard JAR
50-
52+
artifact shadowJar // Only include Shadow JAR
5153
artifact javadocJar
5254
artifact sourcesJar
5355

5456
groupId = 'com.newrelic.labs'
5557
artifactId = 'custom-log4j2-appender'
56-
version = '1.0.5'
58+
version = '1.0.6'
5759

5860
pom {
5961
name = 'Custom Log4j2 Appender'

‎custom-log4j2-appender/publish-jar-legacy.sh‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ cp build-jar.gradle build.gradle
77
# Set variables
88
GROUP_ID="com.newrelic.labs"
99
ARTIFACT_ID="custom-log4j2-appender"
10-
VERSION="1.0.5"
10+
VERSION="1.0.6"
1111
KEY_ID="0ED9FD74E81E6D83FAE25F235640EA0B1C631C6F" # Replace with your actual key ID
1212

1313
# Get the current directory (assuming the script is run from the custom-log4j2-appender directory)

‎custom-log4j2-appender/publish-jar.sh‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ cp build-jar.gradle build.gradle
77
# Set variables
88
GROUP_ID="io.github.newrelic-experimental"
99
ARTIFACT_ID="custom-log4j2-appender"
10-
VERSION="1.0.5"
10+
VERSION="1.0.6"
1111
KEY_ID="0ED9FD74E81E6D83FAE25F235640EA0B1C631C6F" # Replace with your actual key ID
1212

1313
# Get the current directory (assuming the script is run from the custom-log4j2-appender directory)

‎custom-log4j2-appender/publish-shadowJar.sh‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ cp build-shadowJar.gradle build.gradle
66
# Set variables
77
GROUP_ID="io.github.newrelic-experimental"
88
ARTIFACT_ID="custom-log4j2-appender"
9-
VERSION="1.0.5"
9+
VERSION="1.0.6"
1010
KEY_ID="0ED9FD74E81E6D83FAE25F235640EA0B1C631C6F" # Replace with your actual key ID
1111

1212
# Get the current directory (assuming the script is run from the custom-log4j2-appender directory)

‎custom-log4j2-appender/src/main/java/com/newrelic/labs/FlushCallback.java‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
package com.newrelic.labs;
2+
3+
import java.util.List;
4+
import java.util.Map;
5+
6+
public interface FlushCallback {
7+
void onSuccess();
8+
9+
void onFailure(List<Map<String, Object>> failedLogEvents);
10+
}

0 commit comments

Comments
 (0)