Skip to content

Category: Uncategorized

End of a Cycle

Last Friday was my last day working as a Network Automation Engineer at Cargill. Words can’t describe how grateful I am for spending 2020 with them. For those who don’t know, according to Forbes, Cargill is the 2nd largest private US company, primarily in the food and agriculture industry. I was asked many times why would they need network automation engineers?

Cargill runs an immense network and as every other network, it needs to be managed properly. I really cherished the opportunities I got to get exposed to technologies I wasn’t super familiar with such as Jenkins and Batfish while, at the same time, mentoring other engineers on Ansible implementations.

I was really impressed with the overall integrity and high level of excellence of Cargill. Even though they are not a tech company, their container platform is the best I’ve ever worked with so far. Additionally, they are really good at agile practices, ensuring developers deliver their best. More than anything, I cherish the incredible connections I made while working there and I hope I’ll keep for life.

This event and other things got me thinking. This year it’s going to be 5 years since I’ve gotten my Master’s degree in Computer Science and 7 since I made my permanent move to the USA. Many things happened in my life recently that deserve appreciation.

In 2014, I’ve obtained my Bachelors of Science degree in Telecommunication Networks Engineering from Brazil, it took me six years to get that degree because I almost gave up on it on my 5th year. That degree plus other experiences gave me a very strong quantitative background. I did not get exposed to as many technologies as I’d like to but I got exposed to many fundamental problems that are extremely relevant to engineering. In hindsight, I value that experience way more than I did a few years ago. More than that, some of the smartest individuals I met in my life happened during that time.

During my Master’s in the US I dove deep in the research area of Software-Defined Networks. Similarly, initially through Georgia Tech, I had the opportunity to encounter individuals who I admire a lot and once thought were legends such as Vint Cerf, Bob Kahn, Larry Peterson. It was mind boggling to know these guys were real people. Georgia Tech was the environment in which I was most productive and focused in all of my life so far. It’s a tough school, but every minute you spend at Georgia Tech was worth it for me. While at GT I also worked for ONF, that experience itself could lead to a whole new blog post, so I’ll just admit that I’m neglecting them in this specific post. Work wise, ONF has been the most significant job experience for me, so far.

After that, I directed my career to Network Automation and worked for 3 companies, Verizon, PayPal and Cargill. Right now I feel like I owe Verizon, PayPal and ONF their dedicated posts. I might write another one soon. For now, I’m just going to say that I’m quite happy with where all those experiences brought me and I’m very grateful for all I learned so far.

I’m in the last steps of figuring out where I’m going next. I’ll keep you all posted.

Cheers!

Leave a Comment

Where to place Ansible variables?

In this post, I want to discuss a couple of the choices one can have when it comes to placing Ansible variables. The docs for variable precedence state 22 where variables can be defined and the order of precedence that the program applies.

Does that mean you should use all of them? Nope. Although each of one of them has its own use case, I usually vouch for implementation simplicity: less is better.

Here is the list I’d recommend:

  • role defaults
  • inventory group_vars
  • inventory host_vars
  • role vars

I usually do not use other types of variables and I’d not recommend doing so. I reiterate that they can all be useful, but, in my opinion, relying mostly on your inventory variables, reduces the complexity significantly. Which leads to time savings when it comes to troubleshooting and other things.

I remember many times where I spent days working on a bug that was introduced due to a misplaced.

Think of it this way: For every additional layer, you are adding more opportunity for the system to break. For example, one time when we writing playbooks for upgrading OS on a networking device we stipulated that some tasks of the playbook should use a different password, and we thus hardcoded those variables as task vars. For some reason, in production someone was inputting the password variables with more precedence and thus breaking the code. Now, I could argue that the person inputting the password was making a mistake. But in hindsight I’d say the mistake was allowing this complexity to take place at all.

So if you have N layers, you can have O(N^2) possible conflicts. Which means that troubleshooting this can be a nightmare. Ansible developers have experience with this and know how to navigate it, but still why even bother if you could prevent it.

Again, let’s say you wrote your code so that it uses block variables. But then the user wants to overwrite some behavior by setting variables using include_vars? That could mean that you are in uncharted territory, meaning that the code wasn’t tested for this use case…

Role defaults vs Role vars

One pattern that I’ve been using when writing roles using role defaults as an example of how to configure your role. Similar to what is used here:

---
# defaults file for ansible-frr
frr_daemons:
  bgpd: false
  isisd: false
  ldpd: false
  nhrpd: false
  ospf6d: false
  ospfd: false
  pimd: false
  ripd: false
  ripngd: false
  zebra: true

I like to use Role vars when something must be hardcoded. Such as your data model schema. Let’s say the dictionary frr_daemons, must have entries for bgpd. Then I like to put an entry in vars that explicitly defines that and I also add a task on the beginning of my role to check if the inputs are given.

---
# Must have children in variables
Children:
  frr_daemons_: [ 'bgpd' ]

This can get out of hand quick, but usually it works out

Conclusion

In summary, I believe you can place your Ansible variables in the 4 locations listed below. And if you avoid placing variables in any other place you’ll save yourself a lot of troubleshooting time. I’d love to hear if you can think of a usecase to which this example doesn’t suffice.

  • role defaults
  • inventory group_vars
  • inventory host_vars
  • role vars

Please comment below, or PM me on the slack channel from the networktocode folks, my username is castroflavio.

Leave a Comment

Tutorial – What is Infrastructure as Code?

Howdy!! There’s so much network automation content out there, I’ve been feeling like there’s not much value in blogging about it. This is the first post of a of a series with some theory and lab-like tutorials plus code examples of what I believe to be the meat and bones of Infrastructure as Code(IaC).

In the last decade, the industry has been seeing a strong push to adopt software practices towards network engineering. SOME COMPANIES have been successful at that, but I don’t think that’s for everyone. Software development is hard; cost-effective software development is an especially rare beast when you are paying 300K per engineer annually. Okay, so, what?

I claim it’s way more feasible to take a shorter step and aim to implement successful DevOps practices to your networking team than it is to FULLY run your network team like a software team. Infra-as-Code comes in as an enabler for that goal.

Why Infra-as-Code(IaC)?

Infrastructure as Code can simplify and accelerate your infrastructure provisioning process, help you avoid mistakes and comply with policies, keep your environments consistent, and save your company a lot of time and money. Your engineers can be more productive on focus on higher-value tasks. And you can better serve your customers.

https://www.thorntech.com/2018/01/infrastructureascodebenefits/

If IaC isn’t something you’re doing now, maybe it’s time to start!

I’m SOLD! HOW can I get started?

I’m glad I asked! This article defines 6 best-practices to take the most out of IaC:

  1. Codify everything
  2. Document as little as possible
  3. Maintain version control
  4. Continuously test, integrate, and deploy
  5. Make your infrastructure code modular
  6. Make your infrastructure immutable (when possible)

So, this is our lab curriculum, so far:
1. Codify everything: How to use Ansible to automate network provisioning.
3. Maintain version control: Git Tutorial with Ansible
4. Continuously test, integrate and deploy: CI/CD with GitLab
4.1 Continuous test with Batfish and Robot framework

Let me know how this sounds, and if you find this post in the future but I haven’t followed up with it, please make a comment, and I’ll do my best to get it done.

Leave a Comment

Product Management Bootcamp

What’s going on? While working on datacenter networking automation for PayPal this past year, I noticed how much more effective some development teams are when they have a good product manager and that piqued my curiosity. I realized I always spend a good amount of time asking why the work is done before doing it and this is one of the characteristics of a product manager.

Last week, I joined a Bootcamp on Product Management at General Assembly. My goal in joining this Bootcamp is to dive into the subject and understand how product management brings value to the team. Along the way, I hope to acquire some skills to help me be more effective at my job.

In this article, I’m going to list the reading recommendation for the first week of the bootcamp and write a reading summary:

Here is the list of articles:

Good PM/Bad PM by Ben Horowitz and David Weiden – 3.5 stars

Good article that does a good job at densely describing what a good product manager does, and cautions about what a bad product manager does as well.Highlights of a Good PDM:

Clearly defines requirements. The definition is based on research, information and a logical transparent process. This definition empowers engineering to fill the technical gaps, rather than pushing a vision downwards, it builds it’s vision gathering information informally from engineering.
Knows what it takes to make it’s product successful and it defines that in writing.

Doesn’t rest until product vision is consistent across all teams.

What’s Your Problem (Parts 1 and 2 only) by Matt Lavoie – 5 stars

Incredible article that focuses on explaining why defining a problem statement is crucial for the success of an endeavor rather than jumping into solutions as we Engineers usually do. I like that it’s a fun read, yet concise and effective in pushing its point across. Highlights:

With a problem statement, there is no feature creep. There’s a problem and a measurable outcome.
If we believe something will get to that outcome and we can create an experiment to prove it, we should work on it.

By not taking a moment to identify the problem, your implementation won’t be as successful as it could be.

Outcome-driven teams, know when they are successful by utilizing measurement with our output to know that we reached our desired outcome.

The Five Components of a Good Hypothesis by Teresa Torres – 4 stars

Solid reading that describes why one should define good hypothesis for it’s work to be effective. I actually like the format that she criticizes in a later article: How to Improve Your Experiment Design (And Build Trust in Your Product Experiments). Which is the Lean Startup format:

We believe [this capability]
Will result in [this outcome]
We will have confidence to proceed when [we see these measurable signals]

I like it better because its action driven. I can see that depending on the problem her hypothesis template would be more accurate, but the Lean Startup one is lean, it only has the essential to get you moving.

A Product Manager’s Job by Josh Elman – 3 stars

Good read, but unimpressive honestly. It just expands on his vision of the PDM’s job: “Help your team (and company) ship the right product to your users”. Which is true. I think it still needs insights on how this can be achieved for this information to be relevant to me

Product Management vs. Product Marketing by Marty Cagan – 4.5 stars

Great read, and it describes what the author think should be the two complementary roles needed to launch a product. The author states that often the two roles are assigned to the same person even though people usually are focuses in one aspect or the other and that often creates a gap.

Product Management vs. Project Management by Marty Cagan (5/10)

Good read but unimpressive.

This is all for now folks, I’m excited and I’ll try to write a blog post weekly about this new experience.

Leave a Comment