Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Loop error when write log to kafka fail #513

Open
vuhoanghiep1993 opened this issue Jan 6, 2023 · 0 comments
Open

Loop error when write log to kafka fail #513

vuhoanghiep1993 opened this issue Jan 6, 2023 · 0 comments

Comments

@vuhoanghiep1993
Copy link

vuhoanghiep1993 commented Jan 6, 2023

I create a multi writer to file and to kafka as in project https://github.com/vasupal1996/go-app.git

The implement of kafka is : https://github.com/vasupal1996/go-app/blob/master/server/logger/kafkaLogger.go

Problem is when kafka server is down, the diode.poll() loop infinite so it write log a lot

func (kw *KafkaLogWriter) Write(p []byte) (n int, err error) {
	m := kafka.Message{
		Value: p,
	}
	err = kw.WriteMessages(context.Background(), m)
	if err != nil {
		log.Printf("failed to write log message to kafka topic:")
		return 0, err
	}
	return len(p), nil
}

I also want to ask how this infinite for break ,

func (dw Writer) poll() {
	defer close(dw.done)
	for {
		d := dw.d.Next() // -----> when debug, in normal case it out of for loop here, when kafka error, it loop infinite
		if d == nil {
			return
		}
		p := *(*[]byte)(d)
		dw.w.Write(p)

		// Proper usage of a sync.Pool requires each entry to have approximately
		// the same memory cost. To obtain this property when the stored type
		// contains a variably-sized buffer, we add a hard limit on the maximum buffer
		// to place back in the pool.
		//
		// See https://golang.org/issue/23199
		const maxSize = 1 << 16 // 64KiB
		if cap(p) <= maxSize {
			bufPool.Put(p[:0])
		}
	}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant