feat: add net_ldap instrumentation#1587
Conversation
| @encryption = args[:encryption] | ||
| end | ||
|
|
||
| def instrument(event, payload) |
There was a problem hiding this comment.
Do we want more specific attribute mappers here for each of these events?
https://github.com/search?q=repo%3Aruby-ldap%2Fruby-net-ldap+%2Finstrument+%22%2F&type=code
I do not particularly like the practice of dumping json strings where we do not know if the contents of the provided payload have sensitive information in them.
There was a problem hiding this comment.
I agree.
My naive thought was that it would be handled in net-ldap itself, but upon closer inspection I see that this instrumentation can leak e.g. passwords in cleartext.
I'll need some time to figure out which specific attributes to include.
There was a problem hiding this comment.
I think a good starting point would be to capture the name of the operation being performed ie open, read, parse_pdu.
There was a problem hiding this comment.
Spans name is set to the event name, e.g. search.net_ldap (see here for a list of possible span names)
There was a problem hiding this comment.
Sure, but if you look at https://opentelemetry.io/docs/specs/semconv/http/http-spans/ as well as others the span name is usually defined as a templated string which uses the attributes to build the name.
There was a problem hiding this comment.
Thank you, I really appreciate your time and feedback!
I've updated the span name to be LDAP <operation> (modeled after the default Rack instrumentation span naming ). I also found that before I always set span_kind to client; updated this in f0338fc to correctly set it to internal for internal spans as well.
There was a problem hiding this comment.
No worries at all. 🙂 I feel that we have gotten the signal definition into a great shape which is consistent with already defined signals.
My last suggestion based on my last comment would be to add the ldap operation as an attribute ldap.operation.type which would have the same value as used in the span name template.
| tracer.in_span( | ||
| event, | ||
| attributes: attributes.compact, | ||
| kind: :client |
There was a problem hiding this comment.
Do you think LDAP qualifies as RPC? Would using RPC client semantics be right for this?
Would using ECS style events make more sense for these?
There was a problem hiding this comment.
I was inclined to say LDAP qualifies as RPC, but my AI friend does not agree with me. Apparently LDAP itself is not RPC, but it is often combined with RPC (e.g. when joining computers to Windows domains), but we're only instrumenting LDAP in this case. Semconv doesn't specify anything for ldap, and other languages don't seem to have ldap instrumentation.
I'm not sure what to do here.
There was a problem hiding this comment.
Good points. I was also wondering about SemConv as I'm reading through this. I asked the #opentelemetry-semantic-conventions channel on CNCF Slack for advice.
| def initialize(args = {}) | ||
| super | ||
|
|
||
| @instrumentation_service = args[:instrumentation_service] || OpenTelemetry::Instrumentation::Net::LDAP::InstrumentationService.new({ |
There was a problem hiding this comment.
Another thought I have here, what if we added this otel instrumentation as a decorator to the injected service?
There was a problem hiding this comment.
Sorry, I've seem to have missed this earlier. Right now we use the method as prescribed in ruby-net-ldap
kaylareopelle
left a comment
There was a problem hiding this comment.
Hi @scbjans, thanks for opening this PR.
I'm not very familiar with LDAP, so I'll take another pass once I learn a bit more. For now, I left a few comments.
In addition to those comments, would you mind updating the CODEOWNERS file to list you as the owner of this gem? PR assignment for CODEOWNERS is currently broken, but I'm hoping to get it fixed soon.
| gem 'opentelemetry-api' | ||
| gem 'opentelemetry-common' | ||
| gem 'opentelemetry-instrumentation-base' |
There was a problem hiding this comment.
should be able to leave these out, since they're dependencies of the other gems.
| gem 'opentelemetry-api' | |
| gem 'opentelemetry-common' | |
| gem 'opentelemetry-instrumentation-base' |
| tracer.in_span( | ||
| event, | ||
| attributes: attributes.compact, | ||
| kind: :client |
There was a problem hiding this comment.
Good points. I was also wondering about SemConv as I'm reading through this. I asked the #opentelemetry-semantic-conventions channel on CNCF Slack for advice.
|
|
||
| def instrument(event, payload) | ||
| attributes = { | ||
| 'ldap.auth' => auth.except(:password).to_json, |
There was a problem hiding this comment.
Do we need auth given we removed it from db etc?
There was a problem hiding this comment.
I'm not sure why it was removed from db, but I think it's valuable to see which user connects to ldap; e.g. to troubleshoot permissions
There was a problem hiding this comment.
I believe it was in relation to the range of different auth methods/mechanism that could be used ie certificates vs user credentials.
The name Auth doesn't feel scoped enough as it could mean many things. In ldap v3 SASL is supported which opens up additional techniques.
There was a problem hiding this comment.
I'm sorry for the slow turnaround.
In my testsetup, this currently produces: ldap.auth: { method: "simple", username: "<username-of-ldap-bind-user>" }.
This is directly based on the auth hash of the net-ldap gem
Would it be better to have ldap.auth.method and ldap.auth.username? Or shouldn't they be declared under ldap.auth at all?
There was a problem hiding this comment.
If we retain it, then we should go with ldap.auth.method etc
|
@kaylareopelle thanks for your feedback; I updated it in 7264acb @thompson-tomo thank you as well for your feedback, I believe I addressed it in 44a1d54 |
| def instrument(event, payload) | ||
| attributes = { | ||
| 'ldap.auth' => auth.except(:password).to_json, | ||
| 'ldap.base' => base, |
There was a problem hiding this comment.
Should this become ldap.tree.base or similar that way we have potential of adding other properties for the tree.
There was a problem hiding this comment.
I don't foresee that yet, but it's probably best to be futureproof. I'll make that change.
| attributes = { | ||
| 'ldap.auth' => auth.except(:password).to_json, | ||
| 'ldap.base' => base, | ||
| 'ldap.encryption' => encryption.to_json, |
There was a problem hiding this comment.
Not sure of the benefits of this given it could be a hash of properties.
There was a problem hiding this comment.
This is directly based on the encryption hash of the net-ldap gem
In my testsetup, this currently produces: ldap.encryption: { method: "simple_tls", tls_options: { verify_mode: 1 } }. I can imagine this should better fit under
the TLS namespace, however, I don't know how we should define attributes like verify_mode, or ca_file. Might be best to leave it out (at least for now)?
There was a problem hiding this comment.
Agree best to leave it out for now and can be added if need arises but use the tls namespace where appropriate.
| 'ldap.auth' => auth.except(:password).to_json, | ||
| 'ldap.tree.base' => base, | ||
| 'ldap.encryption' => encryption.to_json, | ||
| 'ldap.request.message' => payload.to_json, | ||
| OpenTelemetry::SemConv::SERVER::SERVER_ADDRESS => host || hosts, | ||
| OpenTelemetry::SemConv::SERVER::SERVER_PORT => port, | ||
| OpenTelemetry::SemConv::Incubating::PEER::PEER_SERVICE => instrumentation_config[:peer_service], | ||
| OpenTelemetry::SemConv::NETWORK::NETWORK_PROTOCOL_NAME => 'ldap' | ||
| } |
There was a problem hiding this comment.
Looking at the semconv docs, is there potential for us to set network.transport as well as potentially the network.peer.* atttibutes?
There was a problem hiding this comment.
net-ldap only supports tcp, so I've added that in 97837bc. Since the net-ldap gem abstracts away some connection details, I don't see an easy way to add the network.peer.* attributes
| OpenTelemetry::SemConv::SERVER::SERVER_ADDRESS => host || hosts, | ||
| OpenTelemetry::SemConv::SERVER::SERVER_PORT => port, | ||
| OpenTelemetry::SemConv::Incubating::PEER::PEER_SERVICE => instrumentation_config[:peer_service], | ||
| OpenTelemetry::SemConv::NETWORK::NETWORK_PROTOCOL_NAME => 'ldap' |
There was a problem hiding this comment.
Is there a way to also include the protocol version?
thompson-tomo
left a comment
There was a problem hiding this comment.
Span definition & attribute usage looks good especially as it is consistent with existing sem conv definitions. 👍
Will allow others to validate the coding patterns/structure.
9e45a84 to
93deed5
Compare
45edbb3 to
44f34c0
Compare
|
@arielvalentin @kaylareopelle I believe this PR is ready; could I please get your eyes on this again? |
| module LDAP | ||
| # attribute mapper to redact sensitive keys | ||
| class AttributeMapper | ||
| SENSITIVE_KEYS = %w[userPassword unicodePwd lmPassword msDS-ManagedPassword ntPassword authPassword krbPrincipalKey sambaNTPassword sambaLMPassword].freeze |
There was a problem hiding this comment.
I think it is safer to have an allow list as opposed to a deny list.
This will prevent any new sensitive keys from sneaking in.
There was a problem hiding this comment.
agreed, changed to an allowlist in bce45ff
@jdehaan @Hareramrai could you please have a look if there are any attributes you are missing in attribute_mapper.rb
|
|
||
| # ostruct is not part of the standard library from Ruby4+ | ||
| # from net-ldap 0.20+ it is defined as a gem dependency | ||
| gem 'ostruct' if Gem::Version.new(RUBY_VERSION) >= Gem::Version.new('4') |
There was a problem hiding this comment.
For net-ldap 0.20+ ostruct is defined as a gem dependency.
I'm also fine with updating the compatible block, so that only net-ldap 0.20+ is supported, if that is preferred
There was a problem hiding this comment.
According to this comment the problem was with an old version of rake requiring openstruct. I'll update the Appraisals to keep the compatibility, and move openstruct there
arielvalentin
left a comment
There was a problem hiding this comment.
One last question regarding exception handling. After that I think the 6 month long journey comes to an end.
Thank you (and @kaylareopelle and @thompson-tomo) so much for helping me get this in a better shape! |
|
Hi @scbjans, thank you for your patience with us on this PR. I'd love to get it merged in and released. There's a merge conflict with the release-please-config. Could you take a look at it? |
…, so do not record_exception again.
220f557 to
7be0bd2
Compare
@kaylareopelle no worries, merge conflict is fixed. Failing test seem unrelated. |
Largely based on prior work in #1432 by @Hareramrai and @jdehaan
Since that PR seems stale, I've updated it with some small edits to comply with release management using toys.
Should fix: github.com//issues/669