OSDOCS-12691 [ROSA] New Feature: Log Forwarder #104758
Conversation
openshift-ci
bot
added
the
size/L
|
🤖 Thu Jan 15 20:43:39 - Prow CI generated the docs preview: |
|
There are three sections on the docs preview page that show up highlighted in red. This seems to be some sort or formatting issue. See below for screenshot from one of the three problem sections. 
|
| "Sid": "AllowCentralLogDistributionWrite", | ||
| "Effect": "Allow", | ||
| "Principal": { | ||
| "AWS": "arn:aws:iam::859037107838:role/ROSA-CentralLogDistributionRole-241c1a86" |
There was a problem hiding this comment.
Since we have an IAM role that is assumed by arn:aws:iam::859037107838:role/ROSA-CentralLogDistributionRole-241c1a86, shouldn't the principal here be the ARN of the newly created IAM-role?
There was a problem hiding this comment.
@felixkrohn thanks! I looked at the ARN that we provided in the "creating an iam role" section, and it looks the same as the one here. Unless you see something I don't?
Here's the IAM role ARN:
"Principal": {
"AWS": "arn:aws:iam::859037107838:role/ROSA-CentralLogDistributionRole-241c1a86"
There was a problem hiding this comment.
@jc-berger Yes, it's maybe more of an question regarding architecture than pure docs - feel free to tag anyone else who might be able to chip in here.
From my understanding:
- When in my AWS Account X I provide an IAM role X:Y
- which is then assumed by 859037107838:role/ROSA-CentralLogDistributionRole-241c1a86,
- all further accesses within my Account X would then appear as issued by X:Y
consequently, my bucket policy should control accesses by X:Y and not by 859037107838:role/ROSA-CentralLogDistributionRole-241c1a86?
However, I noticed that the IAM role has never been assumed by anyone in my S3-only case, and audit logging only works when the bucket policy is set up as currently described in the documentation. Which is fine as well, but in that (S3) case the IAM part could be skipped?
(UNLESS I enable S3 bucket encryption using SSE-KMS, in which case this mechanism fails.)
There was a problem hiding this comment.
@felixkrohn thanks for the follow up and explanation!
@willkutler could you help here?
To me, if the ARN lines up with one another, I thought we were good, so not sure where to move forward here.
There was a problem hiding this comment.
@psav may be a better person to ask about this - I'm more familiar with the customer facing API/CLI side rather than how the roles are used. In any case, the way we should document here is the way they are intended to be set up for use with log forwarding, which the existing doc explains correctly
bhardesty
left a comment
bhardesty
left a comment
There was a problem hiding this comment.
This is a great first PR! I did an initial first-pass peer review for about half of the content. I'll re-review once the content is complete.
|
@jc-berger: The following tests failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
| [id="rosa-create-an-iam-role-policy_{context}"] | ||
| = Creating an IAM role and policy | ||
|
|
||
| When you send your logs to a CloudWatch group or S3 bucket, those locations exist outside your control plan. You must create an IAM role and policy so that your log forwarder has the right permissions and capabilities to send these logs to your chosen destination, CloudWatch or S3. |
There was a problem hiding this comment.
@willkutler Aaren and I discussed how we should state the benefit and purpose of creating an IAM role and policy and how it helps with log forwarding.
Am I correct about it here?
There was a problem hiding this comment.
yup I think that sounds right, just change control plan -> control plane
| --assume-role-policy-document file://assume-role-policy.json | ||
| ---- | ||
|
|
||
| .Next steps |
There was a problem hiding this comment.
@bhardesty and @willkutler Aarin and I discussed about making it clear to users that they must set up cloudwatch OR S3, and how they can't do both.
What do you all think about this user flow how as a "next step" users are prompting to selecting one course of action and are given a link to jump them to that specific procedure?
William, am I correct about the benefits of CloudWatch and S3? What I wrote is what I obtained through my research.
There was a problem hiding this comment.
just to clarify, users can only select cloudwatch or s3 for each log forwarder, but they can configure both (and we explicitly allow them to in the ROSA CLI interactive flow) - it just requires two log forwarders, one for s3 and one for CW.
as for the benefits of each that looks ok to me but would have Aaren sign off on it
| // * security/rosa-configuring-the-log-forwarder.adoc | ||
| :_mod-docs-content-type: REFERENCE | ||
| [id="rosa-manage-control-plan-log-forwarding.adoc_{context}"] | ||
| = Managing control plan log forwarding |
There was a problem hiding this comment.
@bhardesty for a reference module, can we have the title start with an action verb? I don't want this to be confused with a procedure, but talking with Aaren, the intent is for users to understand how these commands help them manage their control plane logs.
Also, I note now that I wrote 'plan' instead of 'plane' Iol.
I'll fix that when I address other review comments so there's not too many commits at once!
There was a problem hiding this comment.
| = Managing control plan log forwarding | |
| = Managing control plane log forwarding |
| . Configure your ROSA cluster to access logs from the S3 bucket by applying the following JSON sample: | ||
| + | ||
| [source,json] | ||
| ---- |
There was a problem hiding this comment.
@willkutler did I format this JSON sample correctly?
There was a problem hiding this comment.
is this the file that is passed into --log-fwd-config ? if so that file should be YAML in this format
s3:
s3_config_bucket_name: "my-log-bucket"
s3_config_bucket_prefix: "my-bucket-prefix"
applications: ["cluster-api", "etcd"]
groups: ["api", "authentication", "controller manager", "scheduler"]
note that the applications and groups can both be set to whichever values the user chooses, provided they are a valid application/group name; cluster-api and etcd are just examples
|
|
||
| If you built the control plan logs yourself, you would need to make log forwarding part of the workloads in your cluster. Since the {product-title} log forwarder is in a control plane outside your cluster, it does not take up space or consume resources in your cluster. The {product-title} log forwarder gives your workloads and cluster more bandwidth, making your development processes simpler. | ||
|
|
||
| *IMPORTANT:* When you forward control plan logs, you must decide on what log groups you want to use. Log groups cost money, so the more log groups you have, the more money you spend. See the following table to help you decide what log groups you need before you begin to forward your control plane logs: |
There was a problem hiding this comment.
@bhardesty I know this is long for an assembly.
Aaren wanted a table to explain all the log groups before users begin with the procedures. Figuring that the log groups will cost the users money, I do agree that it's important information to start with and users will want to make a decision before they begin so they don't unintentionally configure more log groups than they need, and therefore, spend money unnecessarily.
Anyway, please let me know what you think of the flow here.
| |==== | ||
| | Log group name| Benefit of that log group | ||
|
|
||
| | API |
There was a problem hiding this comment.
@willkutler I researched all the log groups, so please let me know if my description of their benefits are good.
I don't have to keep the redundant structure of "Helps..." I simply didn't want to guarantee anything and didn't think repetition is bad in a table. But let me know what you think!
There was a problem hiding this comment.
@psav could you take a look at the group descriptions since I believe you originally set up the groups on the api gateway?
@jc-berger a couple thoughts on the structure here:
- "All" is not a group that can be set, users would need to list all the groups to get the logs from all of them
- "Other" is a group containing all logs not in the 4 groups listed here that should probably be mentioned
- do we want to list the applications contained in each group in the docs? maybe more of a question for Aaren
|
|
||
| .Next steps | ||
|
|
||
| After you create an IAM role and policy, you must decide to send your control plan logs to either a CloudWatch log group or S3 bucket: |
There was a problem hiding this comment.
| After you create an IAM role and policy, you must decide to send your control plan logs to either a CloudWatch log group or S3 bucket: | |
| After you create an IAM role and policy, you must decide to send your control plane logs to either a CloudWatch log group or S3 bucket: |
| // | ||
| // * security/rosa-configuring-the-log-forwarder.adoc | ||
| :_mod-docs-content-type: REFERENCE | ||
| [id="rosa-reference-cli-commands.adoc_{context}"] |
There was a problem hiding this comment.
The title for this topic shows up as "Managing control plan log forwarding". Change "plan" to "plane"
| [role="_abstract"] | ||
| With {product-title} you have a control plane log forwarder that is a separate system outside your cluster. You can use the control plane log forwarder to send your logs to either a CloudWatch group or S3 bucket, depending on your preference. | ||
|
|
||
| If you built the control plan logs yourself, you would need to make log forwarding part of the workloads in your cluster. Since the {product-title} log forwarder is in a control plane outside your cluster, it does not take up space or consume resources in your cluster. The {product-title} log forwarder gives your workloads and cluster more bandwidth, making your development processes simpler. |
There was a problem hiding this comment.
| If you built the control plan logs yourself, you would need to make log forwarding part of the workloads in your cluster. Since the {product-title} log forwarder is in a control plane outside your cluster, it does not take up space or consume resources in your cluster. The {product-title} log forwarder gives your workloads and cluster more bandwidth, making your development processes simpler. | |
| If you built the control plane logs yourself, you would need to make log forwarding part of the workloads in your cluster. Since the {product-title} log forwarder is in a control plane outside your cluster, it does not take up space or consume resources in your cluster. The {product-title} log forwarder gives your workloads and cluster more bandwidth, making your development processes simpler. |
|
|
||
| If you built the control plan logs yourself, you would need to make log forwarding part of the workloads in your cluster. Since the {product-title} log forwarder is in a control plane outside your cluster, it does not take up space or consume resources in your cluster. The {product-title} log forwarder gives your workloads and cluster more bandwidth, making your development processes simpler. | ||
|
|
||
| *IMPORTANT:* When you forward control plan logs, you must decide on what log groups you want to use. Log groups cost money, so the more log groups you have, the more money you spend. See the following table to help you decide what log groups you need before you begin to forward your control plane logs: |
There was a problem hiding this comment.
| *IMPORTANT:* When you forward control plan logs, you must decide on what log groups you want to use. Log groups cost money, so the more log groups you have, the more money you spend. See the following table to help you decide what log groups you need before you begin to forward your control plane logs: | |
| *IMPORTANT:* When you forward control plane logs, you must decide on what log groups you want to use. Log groups cost money, so the more log groups you have, the more money you spend. See the following table to help you decide what log groups you need before you begin to forward your control plane logs: |
8 participants
Version(s):
4.20+
Issue:
https://issues.redhat.com/browse/OSDOCS-12691
Link to docs preview:
https://104758--ocpdocs-pr.netlify.app/openshift-rosa-hcp/latest/security/rosa-forwarding-control-plane-logs.html
QE review:
Additional information: