Basic HTTP Auth does not work

[spacepatcher]

Hi!

@lucashenning, it seems that option auth has no effect and because of this Basic HTTP Auth does not work.

The snippet from my Logstash pipeline configuration:

rest {
  request => {
    url => "http://<FQDN>/"
    method => "get"
    auth => {
      user => "user"
      password => "pass"
    }
    params => {
      "v" => "%{[event_data][Ip]}"
    }
  }
  json => true
  target => "[event_data][Info]"
}

I expect the Basic HTTP Auth header in an HTTP request (Authorization: Basic dXNlcjpwYXNz) generated by logstash-filter-rest, but it is missing!

Here is the real HTTP request:

GET /?v=8.8.8.8 HTTP/1.1
Connection: Keep-Alive
Content-Length: 0
Host: <FQDN>
User-Agent: Manticore 0.6.4
Accept-Encoding: gzip,deflate

[lucashenning]

Okay. Thanks for the bug report. I haven't looked into this but as a quick workaround, have you tried:

url => "http://user:pass@<FQDN>/"

[spacepatcher]

@lucashenning, yes, I tried. The Basic HTTP Auth header is also missing...

[spacepatcher]

@lucashenning, could you please take the time to this bug? I think this is probably a really serious problem.

[spacepatcher]

I assume the reason is in a server that has a poor Basic Auth implementation, and not in a client. The server response on the first request from the client:

HTTP/1.1 401 Unauthorized
Server: nginx/1.15.5
Date: Thu, 20 Dec 2018 10:39:16 GMT
Content-Type: text/plain; charset=UTF-8
Content-Length: 84
WWW-Authenticate: ""

Here you can see that WWW-Authenticate header is empty, but it should contain the required method of authentication.

The only way to force authentication is to include the required Basic Auth header in the initial client request. This behavior has been implemented by default in logstash-mixin-http_client since v5.1.0 (logstash-plugins/logstash-mixin-http_client@cc54fc1#diff-a289f11e8a926bdddbf1887fcb11d9beR135). The current version of logstash-filter-rest uses logstash-mixin-http_client v6.0.1, and the reason for the lack of Basic Auth header in the first client request is completely unclear to me.

Fortunately I found an easy way to get around this problem. The only need to craft the required Basic Auth header and include it in section headers:

headers => {
  "Authorization" => "Basic <base64>"
}