Network Brouhaha Networking, Cloud, Automation, Infrastructure, Containers and General Geekery http://www.networkbrouhaha.com/ I'm joining Rubrik <p>This post will be short and sweet. I’m joining Rubrik as a Technical Marketing Engineer, focusing on Networking and Security.</p> <p>Why:</p> <ul> <li>To challenge myself and learn new things</li> <li>To contribute some of my networking (and other) knowledge</li> <li>To focus my energy into a solid product that I believe in</li> <li>To work with some incredibly bright minds</li> <li>Rubrik has a growing customer base with increasingly complex networks</li> </ul> <p>Leaving SIS was a difficult decision. I made some great friends there and genuinely hope those friendships continue. I don’t have anything to say but kind words for the SIS folks, and this new direction isn’t because of anything negative happening there. Simply put, an opportunity to work at Rubrik is one that is too good to pass up. They are disruptive in their market, and by my appraisal they are doing things the right way. No one has anything but good things to say about their leadership, and they have a compelling story about their product.</p> <p>I can’t wait to see what the next few months will bring. I’m excited to learn everything about Rubrik and get to know my new team members. Unfortunately this means that I’ll have to pause work on my “Hybrid Home Lab” setup, but I will continue that effort as soon as I can.</p> Mon, 22 Oct 2018 00:00:00 +0000 http://www.networkbrouhaha.com/2018/10/im-joining-rubrik/ http://www.networkbrouhaha.com/2018/10/im-joining-rubrik/ VMworld 2018 Recap <p>My journey to VMworld 2018 began in an unexpected way - a <a href="https://twitter.com/hcmccain/status/1023994969810460674">tweet from Chris McCain</a>.</p> <blockquote> <p>u a CCIE? Apply for a full VMworld pass.</p> </blockquote> <p>What did I have to lose? Never mind that VMworld was only a month away, and I had no approval to actually travel to Las Vegas. I filled out a little survey about how much I love NSX, and pressed submit. <em>Fast forward a week, a wild email appears:</em></p> <blockquote> <p>Thanks for filling out the application, I’d love to offer you the NSX Mindset scholarship to VMworld!</p> </blockquote> <p>Uh oh. That is not what I expected. After a good amount of scrambling on my part, and some very gracious actions by my employer, I was approved for travel and lodging in Las Vegas for VMworld 2018. What an exciting an unexpected turn of events! I registered, made arrangements, and started prepping to head to a conference I never expected to attend. With CiscoLive fresh on my mind, I made a concerted effort to keep my schedule realistic. I definitely wanted plenty of time to meet new people, hit some hands on labs, and some down time so I didn’t exhaust myself. I exercised self control while scheduling sessions, but it was not easy. There was a long list of options that piqued my interest.</p> <p>It’s no secret that my roots are in networking. I’ve spent plenty of time in the compute and storage silos, and I attend my local VMUG, but I am an “outsider” when it comes to the #vCommunity. I expected that VMworld was a lot like CiscoLive in terms of form and function. I found this to be mostly true, but there are some differences that I will call out throughout this post.</p> <p>August crept by slowly, but the time finally came to board a plane bound for Las Vegas. After checking into my hotel and grabbing my VMworld badge, it was time for <a href="https://blog.vmunderground.com/opening-acts-2018/">Opening Acts</a>, followed by <a href="https://blog.vmunderground.com/vmunderground-2018/">VMunderground</a>. This was the first of several differences I noticed between VMworld and CiscoLive. If VMworld was a planet, it would have several orbiting moons that represent all the community events happening in conjunction. vDodgeball, vSoccer, vFit runs, vBeards gatherings - there is something for everyone. From what I can tell, these all have roots in VMUG (or vBrownbag). VMware made a smart decision supporting and empowering VMUG leaders. It has spawned a vibrant community, and it sets VMworld apart from other events.</p> <p>Opening Acts was a great way to kick off VMworld. The panel on <a href="https://www.youtube.com/watch?v=D2CMVJQPZio">“Beating IT Burnout”</a> was a highlight, and it was fun watching my friend <a href="https://twitter.com/tbgree00">Thom</a> up on stage. <a href="https://twitter.com/MindfulAlicia">Alicia Preston</a> spoke about practicing mindfulness to combat burnout. This presentation spawned several other hallway conversations throughout the week. If you missed it, take the time to watch. VMunderground was also a great time, and I got the opportunity to meet several folks that I would continue to see at blogger tables and VMTN area. I definitely recommend this event for anyone that is new to VMworld.</p> <h3 id="sessions">Sessions</h3> <p>Overall the session content was very good, and I was surprised at the depth of the networking material. In general, I found the sessions to be a bit more technical at CLUS than VMworld, but not by much. One thing I missed from CLUS was having access to a copy of the slides for each session. There were several times a presenter blew past a slide that I wanted to digest a bit more. It also keeps people from feeling like they have to snap a picture of every slide. Here are my highlights:</p> <ul> <li><a href="https://videos.vmworld.com/searchsite/2018/videoplayer/18995">NSX Mindset: Clouds Collide, Opportunity Strikes (NET1919BU)</a> - This is not a technical talk, but I’d recommed it to anyone working in IT. Chris McCain is a fantastic presenter and could probably work the motivational speaker circuit.</li> <li><a href="https://videos.vmworld.com/searchsite/2018/videoplayer/20207">Kubernetes NSX-T Deep Dive (NET1677BU)</a> - I’ve spent plenty of hours trying to detangle networking in Kubernetes. This presentation lays out k8s topics and constructs in an easy to understand way, and makes a great case for NSX-T as one of the best ways to “do networking” in Kubernetes.</li> <li><a href="https://videos.vmworld.com/searchsite/2018/videoplayer/22674">Next-Generation Reference Design with NSX-T Data Center: Part 1 (NET1561BU)</a></li> <li><a href="https://videos.vmworld.com/searchsite/2018/videoplayer/22675">Next-Generation Reference Design with NSX-T Data Center: Part 2 (NET1562BU)</a></li> <li><a href="https://videos.vmworld.com/searchsite/2018/videoplayer/23018">VMware Cloud on AWS with NSX: Use Cases, Design, and Implementation (NET1327BU)</a> - Good overview of networking in VMWonAWS, plus a preview of things to come with NSX-T support.</li> </ul> <h3 id="keynotes-and-announcements">Keynotes and announcements</h3> <p>Honestly, I don’t really care about keynotes at conferences. The only ones I’m truly interested in are the non-technical ones, a la Michio Kaku &amp; Amy Webb at CLUS, and Malala Yousafzai at VMworld. All of the announcements are already well covered, so I’m not going to generate yet another list. I was absolutely thrilled at the opportunity to hear Malala speak, and I give VMware major credit for bringing her to speak, along with committing to supporting her charity. There were some grumbles about the increased security, but in my opinion it was all worth it. I am so inspired by this young woman and her commitment to fighting for education for girls everywhere. Someone - I’m not saying who - recorded her talk on Periscope, and you can watch <em>here</em>. Thinking it still gives me all the feels.</p> <h3 id="parties">Parties</h3> <p>Maybe it’s because VMworld is in Las Vegas, but it would be an understatement to say that there were lots of parties going on. My MO for conferences is to treat them like work. I’m there to learn, and my employer is paying for me to be there. However, there were a few baller parties that are worth mentioning.</p> <ul> <li>Rubrik had the party of the week in my option. RUN-DMC <em>and</em> The Roots?! It was non-stop awesome and I danced my butt off. I have been a fan of The Roots since 1996, and I had only seen them live once. I made my way to the front of the stage and enjoyed a once-in-a-lifetime show. RUN-DMC was also great and Jam Master Jay’s son is a hell of a turntablist. Kudos to Rubrik for throwing a great party. <a href="https://www.youtube.com/playlist?list=PLmyCQ1p5hbAgWITKwFW6HEYAGk21b7OPQ">Here are a couple videos I took from the party</a></li> <li>VMfest was, in my opinion, a fun time. Several people I talked to skipped the party altogether. I’ve read comments from many people that thought it was terrible. <a href="http://www.royalmachinesmusic.com/home/">Royal Machines</a> was an unpopular choice for a band - I was disappointed when I saw the announcement. If this wasn’t my first VMworld I may have skipped the party as well, but I decided to go into it with an open mind. When I walked in, there were <em>long</em> lines for food trucks scattered around the entrance area. I have no idea why people were waiting as there was food available in several other places. I never had to wait in line for a drink all night. The theme was four different environments: tropical, desert, jungle, aquatic. Maybe this turned people off - I thought it was an original idea and the decorations were well done. Royal Machines were a pleasant surprise. I’m a sucker for a good cover band, and it was a fun show. They completely embraced the ridiculousness of who they are. Dave Navarro is a rock god - it was a pleasure to watch him play. Mark McGrath understands that everyone thinks he’s a joke, and he is still willing to get out on the stage and give it his all. He gets my respect for that. Macy Gray covering Radiohead: Awesome. Sebastian Bach covering Ozzy: Awesome. Robin Zander in general: Awesome. Surprise appearance by DMC: Awesome. <a href="https://www.youtube.com/playlist?list=PLmyCQ1p5hbAgazhLfv2Lvu5iwhEPIrKd3">Videos from the show</a> / <a href="http://www.royalmachinesmusic.com/home/latest/events/vmware-las-vegas/">Setlist</a></li> <li>The NSBU threw an “NSX Mindset” party at the <a href="https://1923lv.com">1923 Bourbon Bar</a>. The place was packed, and rockin’. I truly wish I could have stayed longer, but I did not want to experience FUTURE:NET with a hangover. I did the responsible (i.e. boring) thing and slipped out early.</li> </ul> <h3 id="futurenet">FUTURE:NET</h3> <p>Future:net is a one day “conference within a conference”, described as a “discussion on the future of networking with industry leaders and visionaries”. It is invite-only, and I was lucky to receive an invitation along with my scholarship. I first heard about this event on <a href="https://packetpushers.net">Packet Pushers</a>, and I was immediately intrigued. Of everything I had scheduled at VMworld, I was most excited for this event, and it did not disappoint. The event took place all day Thursday, and there was a welcome reception Wedensday evening. I considered skipping the reception, and I’m glad I walked over to The Four Seasons instead of taking a nap. The first person I met leads networking teams at Google. Not long after that, Pat Gelsinger showed up. I was standing right beside him as he and Greg Ferro made a bet about the SD-WAN industry.</p> <blockquote> <p>I just made a SD-WAN bet with @pgelsinger that NSX Velocloud and Cisco Viptela will NOT have 70% market share by this time next year. Tell me I am wrong ? https://twitter.com/etherealmind/status/1035011002855636992</p> </blockquote> <p class="center"><img src="https://networkbrouhaha.com/resources/2018/09/pgelsinger.jpg" alt="" height="50%" width="50%" /></p> <p>Thursday the conference kicked off with breakfast and a live recording of a Packet Pushers podcast, which was a real treat to watch. I have been a loyal listener for many years, but I had never gotten the chance to meet Greg, Ethan and Drew. After breakfast, the presentations began, and the first presenter was a professor from Cornell discussing blockchain. Of every presenter on the agenda I was least excited for this talk - I feel like we’ve all heard more than enough about blockchain already. I was completely wrong, and it may have been my favorite talk of the day. <a href="https://twitter.com/el33th4xor">Emin Gun Sirer</a> delivered fascinating talk about why blockchain as a technology is much more interesting than cryptocurrencies.</p> <p>I live tweeted the event and this blog is already long enough, so you can see my thoughts and others here: <a href="https://twitter.com/search?f=tweets&amp;vertical=default&amp;q=%23futurenet18&amp;src=typd">#FutureNET18</a>. You can also find coverage in <a href="https://packetpushers.net/podcast/weekly-show-406-updates-and-introspection/">Packet Pushers Weekly Episode 406</a> and <a href="https://packetpushers.net/podcast/network-break-200-vmware-navigates-multicloud-perils-and-opportunities/">Network Break 200</a>. I will try to write some more words about this event later - it really deserves its own blog post. Needless to say I was honored to attend and it was one of the highlights of my week.</p> <p class="center"><img src="https://networkbrouhaha.com/resources/2018/09/packetpushers.png" alt="" height="50%" width="50%" /></p> <h3 id="closing-thoughts">Closing thoughts</h3> <ul> <li>As I mentioned, I’m not going to regurgitate all of the announcements from VMworld. Here’s a few links if you still need to catch up. <ul> <li>https://www.vmware.com/products/whats-new.html?src=so_5a314d05e49f5&amp;cid=70134000001SkJn</li> <li>https://anthonyspiteri.net/vmworld-2018-recap-part-1-major-announcement-breakdown/</li> <li>https://anthonyspiteri.net/vmworld-2018-recap-part-2-community-and-veeam-recap/</li> </ul> </li> <li>VMworld has a little ways to go in terms of organization. Compared to CLUS, registration was a hot mess. CiscoLive is a larger conference, and Cisco clearly throws a <em>lot</em> of resources at it. There are some other small things like putting tables in the breakout room that I missed. Would this stuff prevent me from coming again? Probably not. VMware does a very good job with this conference, but they could take a couple pages out of Cisco’s playbook.</li> <li>There was a question thrown out in the Packet Pushers slack: If you went to VMworld this year, would you go again? My answer is probably. I’m not sure if it’s an event that I would need to hit every year, but I really enjoyed my experience. The only thing that bothered me was the location. I am not a fan of Las Vegas. Everything is too expensive. Everything is over the top. There are times when I’m mildly amused, but they are few and far in-between. I am <em>not</em> the morality police and I am not interested in judging anyone, but being in Vegas pushes me to the edge. It makes me feel icky. I’ve made no decision on if I’ll request to go to San Francisco in 2019, but I’ll seriously consider it.</li> <li>Some genius at DEF CON was handing out “blockchains” - miniature cinder blocks on a dogtag chain. I found this to be incredibly punny, so I gathered the necessary materials and brought some with me to Vegas. I figured it would be a fun way to break the ice and meet new people, and I was not wrong. Everyone loved them, and I met so many people that I would not have met otherwise. I wish I knew who came up with the original idea so I could give him/her credit. Having something fun to share is an awesome way to meet people, especially if you’re a newcomer. If I’m already thinking about ways to expand on this idea if I make it to VMworld in San Francisco.</li> </ul> <p class="center"><img src="https://networkbrouhaha.com/resources/2018/09/blockchain.png" alt="" /></p> <ul> <li>If you’re in Vegas and you don’t get a meal at Hash House a Go Go, you’re losing at life.</li> </ul> <p class="center"><img src="https://networkbrouhaha.com/resources/2018/09/hashhouse.png" alt="" height="40%" width="40%" /></p> Thu, 13 Sep 2018 00:00:00 +0000 http://www.networkbrouhaha.com/2018/09/vmworld-2018-recap/ http://www.networkbrouhaha.com/2018/09/vmworld-2018-recap/ Hybrid Home Lab Pt. 1 <p>Over the last few weeks I’ve been working on standing up my version of a “real” lab. I’ve got enough information together to start putting together some blog posts, so let’s dive right in. Previously, my home lab was just a custom built linux server with plenty of memory and software RAID. This was enough to do some small-scale network labs and run the few applications I needed, but it really doesn’t qualify as a true home lab. There’s no way for me to work with a vSphere or KVM cluster, let alone NSX-v or NSX-T. I’ve laid out a few goals for my “Hybrid Home Lab”:</p> <ul> <li>On Prem Resources <ul> <li>2x UCS C220 M3</li> <li>Re-purpose existing server as a home NAS <ul> <li>Utilize hardware RAID and serve LUNs via iSCSI or NFS</li> </ul> </li> <li><a href="https://www.ubnt.com/edgemax/edgeswitch-16-xg/">Ubiquiti EdgeSwitch 16XG</a></li> <li><a href="https://www.ubnt.com/edgemax/edgerouter-poe/">Ubiquiti EdgeRouter PoE</a></li> <li>Purpose: Compute Virtualization Lab (vSphere or KVM), Network Virtualization Lab (NSX-V, NSX-T, EVE-NG, VIRL, GNS3), Kubernetes backup cluster</li> </ul> </li> <li>Cloud Resources <ul> <li>Hosted in <a href="https://www.vmware.com/products/vcloud-director.html">vCloud Director</a></li> <li><a href="https://rancher.com/blog/2018/2018-05-01-rancher-ga-announcement-sheng-liang/">Rancher 2.0</a> Kubernetes cluster</li> <li><a href="https://opnsense.org">OPNsense</a> firewall <ul> <li>Also provides <a href="https://www.zerotier.com">ZeroTier</a> VPN/SD-WAN and <a href="https://haproxy.org">HAproxy</a> load balancing</li> <li>Replaces NSX edge in vCD</li> </ul> </li> <li><a href="https://www.gluster.org">Gluster</a> for persistent Kubernetes storage</li> <li>Purpose: Learn Kubernetes, deliver applications independent of on prem resources, test OPNsense as a “cloud router” and ZeroTier for hybrid cloud scenarios <ul> <li>Applications I’ll try to run: Gitlab, Netbox, Zabbix, Grafana, MariaDB/Postgres, StackStorm, other automation tools, and custom</li> </ul> </li> </ul> </li> </ul> <p>I will be publishing detailed blog posts on the setup of these components - stay tuned!</p> <h1 id="but-why">But, Why?</h1> <p class="center"><img src="https://networkbrouhaha.com/resources/2018/08/ytho.jpg" alt="" height="35%" width="35%" /></p> <p class="center">(my daughter approves the use of this meme)</p> <p><strong>Why vCD?</strong> I have access to a vCD lab at work. I have to keep a small footprint, but this is much more economical than using another cloud provider. We’ve run vCD at my <a href="https://thinksis.cmo">day job</a> for quite a while, and I’ve become fond of it. It’s come a <em>long</em> way since we initially deployed it, and it continues to improve. <a href="https://twitter.com/search?f=tweets&amp;vertical=default&amp;q=%23LongLiveVCD">#LongLiveVCD</a></p> <p><strong>Why Rancher?</strong> This is another product that we’re using at work, so I have some motivation to learn it. It definitely is “training wheels” for Kubernetes, and I’m already getting the itch to experiment with vanilla Kubernetes or OpenShift. For now it does what I need it to, and it’s not terribly difficult to take all my YAML files and load them in another Kubernetes cluster later.</p> <p><strong>Why are you running stateful applications in Kubernetes?</strong> I understand that Kubernetes is mainly for stateless applications and microservices, but it does support stateful workloads. This is a lab, and sometimes it is fun to push the limits.</p> <p><strong>Why Gluster?</strong> Persistent storage in Kubernetes is a PITA if you’re not using one of the major cloud providers, or leveraging storage that provides a Kubernetes plugin. <a href="https://github.com/heketi/heketi">Heketi</a> provides an API interface for GlusterFS that Kubernetes can leverage. I’ll provide more information in a later blog post, but this was the easiest way to provide redundant persistent storage for my Rancher cluster.</p> <p><strong>Why OPNsense?</strong> Yes, vCD provides an NSX edge. In vCD 9.1, it is full featured and suitable for most workloads. I’m a network nerd so this is one of the areas where I want more flexibility than what NSX can provide. The <a href="https://opnsense.org/about/features/">feature list</a> for OPNsense is impressive, and most importantly for me, it has built in support for ZeroTier.</p> <p><strong>Why ZeroTier?</strong> Please see my previous post on <a href="/2018/03/vcd-terraform-example/">cloud automation</a>. Future posts will go into more detail on this as well.</p> <h1 id="show-me-the-diagram">Show me the diagram</h1> <p>IP addresses have been changed to protect the innocent.</p> <p class="center"><a href="https://networkbrouhaha.com/resources/2018/08/hybrid_lab_diagram.png" height="75%" width="75%"><img src="https://networkbrouhaha.com/resources/2018/08/hybrid_lab_diagram.png" alt="hybrid lab diagram" /></a></p> <p class="center">(Click to embiggen)</p> Tue, 21 Aug 2018 00:00:00 +0000 http://www.networkbrouhaha.com/2018/08/hybrid-home-lab-pt1/ http://www.networkbrouhaha.com/2018/08/hybrid-home-lab-pt1/ CLUS 2018 recap <p>For the first time in seven years, I had the opportunity to travel to Cisco Live 2018 in Orlando, FL. In this belated blog post, I’ve got a few thoughts, a few tips, and a bit of geeking out.</p> <p>There’s a thrill to registering for Cisco Live: scheduling sessions, RSVPing to party invites, planning to meet friends, and booking flights. The most important part, by far, is creating a reasonable schedule. CLUS is a marathon, not a sprint, and you have to be careful to not overburden yourself. I was at packed 8:00am sessions every day but Thursday, and up fairly late most nights. There is simply too much to do. Below is a list of sessions I attended, to get an idea of my week.</p> <ul> <li>[BRKSDN-2262] <a href="https://www.ciscolive.com/global/on-demand-library/?search=BRKSDN-2262#/session/1516099602451001CqMa">Open Source for Networking: The FD.io/VPP example</a></li> <li>[DEVNET-1293] <a href="https://www.ciscolive.com/global/on-demand-library/?search=DEVNET-1293#/session/1509733975288001YGm4">Cisco UCS Automation and orchestration with Ansible</a></li> <li>[BRKDCN-2035] <a href="https://www.ciscolive.com/global/on-demand-library/?search=BRKDCN-2035#/session/1509501687106001POqy">VXLAN BGP EVPN based Multi-Site</a></li> <li>[DEVNET-2644] <a href="https://www.ciscolive.com/global/on-demand-library/?search=DEVNET-2644#/session/15111940816080019wR4">Open Network Automation Platform</a> (ONAP)</li> <li>[BRKDCN-3040] <a href="https://www.ciscolive.com/global/on-demand-library/?search=BRKDCN-3040#/session/1509501655684001PLO4">Troubleshooting VxLAN BGP EVPN</a></li> <li>[DEVNET-1296] <a href="https://www.ciscolive.com/global/on-demand-library/?search=DEVNET-1296#/session/1510584364275001jLdB">Building a NetDevOps CICD Pipeline with OpenSource</a></li> <li>[BRKSDN-2115] <a href="https://www.ciscolive.com/global/on-demand-library/?search=BRKSDN-2115#/session/1512002243477001x6sa">Introduction to Containers and Container Networking</a></li> <li>[BRKDCN-3001] <a href="https://www.ciscolive.com/global/on-demand-library/?search=BRKDCN-3001#/session/1512769713770001R5Fc">Leveraging Micro Segmentation to Build Comprehensive Data Center Security Architecture</a></li> <li>[BRKRST-3310] <a href="https://www.ciscolive.com/global/on-demand-library/?search=BRKRST-3310#/session/1518011397038001CXX2">Troubleshooting OSPF</a></li> <li>[BRKCLD-3440] <a href="https://www.ciscolive.com/global/on-demand-library/?search=BRKCLD-3440#/session/1511296161600001A5Dh">Multicloud Networking – Design &amp; Deployment</a></li> <li>[BRKDCN-2125] <a href="https://www.ciscolive.com/global/on-demand-library/?search=BRKDCN-2125#/session/1509501687216001Pex1">Overlay Management and Visibility with VXLAN</a></li> <li>[DEVNET-1365] <a href="https://www.ciscolive.com/global/on-demand-library/?search=DEVNET-1365#/session/1499457537273001QPDr">DevNet Workshop- Vagrant Up for the Network Engineer (NX-OS, IOS-XE, IOS-XR)</a></li> <li>[DEVNET-2076] <a href="https://www.ciscolive.com/global/on-demand-library/?search=DEVNET-2076#/session/1510880880567001k3i2">Continuous Integration and Testing for Networks with Ansible</a></li> <li>[BRKSEC-2010] <a href="https://www.ciscolive.com/global/on-demand-library/?search=BRKSEC-2010#/session/1509501665659001Pw8M">Talos Insights: The State of Cyber Security</a></li> <li>[KEYGEN-1003] <a href="https://www.ciscolive.com/global/on-demand-library/?search=KEYGEN-1003#/session/1520266383574001fJzL">Closing Keynote: What Science Can Tell Us About Our Future</a></li> </ul> <p>Here is the approach I took to building my schedule:</p> <ul> <li>I went through the course catalog, filtering by technology, and marked every interesting course as a favorite. All favorites are saved, so you can go back and watch recordings for sessions you missed once they’re posted.</li> <li>I noted 5-6 “must attend” sessions, and registered for them as soon as registration opened.</li> <li>Filtering by time slot and favorite sessions, I filled up the rest of my schedule. I didn’t worry about leaving time for lunch at this stage.</li> <li>After some internal deliberation, I dropped between 1/3rd and 1/4th of the courses I’d registered for. This provided time to eat, rest, socialize, and attend some of the “meatspace only” opportunities (DevNet, Walk-in Self Paced Labs, Tweetups etc.)</li> </ul> <p>I knew I’d made good picks when I walked into my first session and sat down behind Terry Slattery and Wendell Odom. My favorite session was <a href="https://www.ciscolive.com/global/on-demand-library/?search=BRKRST-3310#/session/1518011397038001CXX2">Troubleshooting OSPF</a>, by Nick Russo. The room was packed, and Nick put on a master class. If you missed it, do yourself a favor and watch it now. You don’t need to be an OSPF guru to keep up, but I’m willing to bet that even the most seasoned CCIE R&amp;S will gain something from this session. Overall the session content across the board was top notch, with only a couple sessions that I found mildly disappointing at worst.</p> <p>Almost every session recording is <a href="https://www.ciscolive.com/global/on-demand-library/">posted online</a>, so there is no reason to have Cisco Live session FOMO. Most of us go to CLUS to learn the latest and greatest in our chosen technology stacks, but I find far greater value in the human connections I formed. I’m an extrovert, so being surrounded by a throng of people gives me energy. As I walked down the halls I would look around and think to myself, “Yes, these are my people!”</p> <p>I made a concerted effort to connect with as many online friends and personal inspirations as I could. Here’s a incomplete list of folks I was either able to meet or learn from: <a href="https://rule11.tech">Russ White</a>, <a href="https://twitter.com/bcjordo">Jordan Martin</a>, <a href="https://twitter.com/SharpNetwork">Eyvonne Sharp</a>, <a href="https://www.netcraftsmen.com/team/terrance-slattery/">Terry Slattery</a> (plus many other NetCraftsmen I sat in sessions with), <a href="https://twitter.com/Wendellodom">Wendell Odom</a>, <a href="https://twitter.com/ScottMorrisCCIE">Scott Morris</a>, <a href="https://twitter.com/CCIE21921">Lukas Krattiger</a>, <a href="https://twitter.com/hfpreston">Hank Preston</a>, <a href="https://twitter.com/jedelman8">Jason Edelman</a>, <a href="https://twitter.com/nickrusso42518">Nick Russo</a>, <a href="https://twitter.com/danieldibswe">Daniel Dibb</a>, <a href="https://twitter.com/dmfigol">Dmitry Figol</a>, <a href="https://twitter.com/kmcnam1">Katherine McNamara</a>, <a href="https://www.networkingwithfish.com">Denise Fishburne</a>, <a href="https://www.linkedin.com/in/humphreycheung/">Humphrey Cheung</a>, <a href="https://twitter.com/theLANtamer">Quentin Demmon</a> and <a href="https://twitter.com/showipintbri">Tony Efantis</a>, not to mention all the fine folks I met from <a href="https://www.meetup.com/routergods/">RouterGods</a>. This is a prolific group of networkers. If you want to improve yourself, what better way is there than learning from people like this? I’m also a believer in spreading gratitude, so I made sure to personally thank folks that had helped me grow technically and professionally. Every single person I thanked seemed genuinely appreciative to hear it. There’s never any harm in spreading the love!</p> <p>My only regret is that I did not hunt down <a href="https://twitter.com/Drew_CM">Drew Conry-Murray</a>, as I am an avid <a href="https://packetpushers.net">Packet Pushers</a> listener and I love <a href="https://packetpushers.net/series/network-break-podcast-post/">The Network Break</a>. Hopefully I can remedy this next year!</p> <p>I have to give special attention to the <a href="https://developer.cisco.com">DevNet Zone</a>, and the folks that put it all together. This area was filled with some of the best content of the conference. Network Automation, Programming, APIs and the future of Networking in general was on full display. There were hands-on labs and experts willing to whiteboard anything you wanted to discuss. Watching Wendell Odom geek out like the rest of us as Hank Preston presented on NetDevOps was a particularly cool moment.</p> <p class="center"><img src="https://networkbrouhaha.com/resources/2018/07/networkcicd.png" alt="" height="50%" width="50%" /></p> <p>You’ll notice from the list of sessions above that I only attended one keynote. There were DevNet sessions that I wanted to attend instead, and the keynotes are posted online, so it wasn’t a tough decision. The closing keynote, featuring <a href="https://amywebb.io">Amy Webb</a> and <a href="http://mkaku.org">Dr. Michio Kaku</a> is a different story. By Thursday I was running on fumes, so I took the day easy. About an hour before the closing keynote, I made my way towards the entrance and saw a huge line had already formed. I had no interest in standing for an hour, so I found an empty seat nearby and waited for the doors to open. For some reason they didn’t open the doors where folks had queued - they opened the doors <em>directly</em> behind the seat I was sitting in. I was surprised and felt bad for the people that had been waiting in line, but I’m no dummy. I grabbed my stuff, walked in, and got seated in the front row almost directly in front of the stage. Talk about good luck! To top it off, as I was sitting there, one of my tweets was flashed up on the uber-displays. It was an amazing and surreal way to end CLUS. Both Amy and Michio gave great keynotes to wrap up CLUS.</p> <p class="center"><img src="https://networkbrouhaha.com/resources/2018/07/clustweet.png" alt="" height="75%" width="75%" /></p> <h2 id="closing-thoughts-and-tips">Closing thoughts and tips</h2> <p>I had a great time at Cisco Live 2018. It was so fulfilling to meet and hang out with everyone, learn new things, explore the DevNet Zone/World of Solutions, and attend several great parties. I will admit to feeling somewhat overwhelmed the whole time I was there. There is something bright and shiny to grab your attention at every turn. Keeping up with twitter is a job within itself, and the Cisco Social Media team really deserves kudos for the great job they do during CLUS. However, I cannot disagree with anything Tom Hollingsworth wrote in his <a href="https://networkingnerd.net/2018/06/22/finding-value-in-cisco-live-2018/">Cisco Live Recap</a>. CLUS is a great event, but there will always be ways to improve and provide better value. In the end, like most things, you will get out of it what you put into it.</p> <p>Here’s a few random tips to wrap up this post</p> <ul> <li>Take breaks - you will need time to decompress.</li> <li>Stay hydrated.</li> <li>Come prepared to learn a lot, and keep a notebook handy. You may find yourself wanting to take notes when least expected.</li> <li>Put yourself out there. Go out of your way to introduce yourself to peers in sessions, during meals, and at parties. Bring business cards.</li> <li>If you’re social, hit the Tweetups. This is a great place to meet people and network.</li> <li>Go easy at the parties. You’ll do yourself no favors by trying to make it through the next day hungover.</li> <li>HAVE FUN.</li> </ul> <p class="center"><img src="https://networkbrouhaha.com/resources/2018/02/drink_route_tr.png" alt="" height="25%" width="25%" /></p> Mon, 23 Jul 2018 00:00:00 +0000 http://www.networkbrouhaha.com/2018/07/CLUS-recap/ http://www.networkbrouhaha.com/2018/07/CLUS-recap/ Checking in <p>Things have been a little quiet around here, so I wanted to put up a quick post recapping what I’ve been up to as well as things coming down the pike. I’ve been on quite a run since my last post. The main time sink was passing several tests to keep our Cisco “Advanced Data Center Architecture Specialization” before it expired on 7/1. The upside is the material was <em>mostly</em> the same as what I studied for my CCIE DC. The downside is that my CCIE DC is useless for this specialization. Yes, I’m a little bitter. Here’s are the tests that I had to speed run.</p> <ul> <li><a href="https://learningnetwork.cisco.com/community/certifications/ccnp_data_center/dcii/exam-topics">300-165 DCII (Implementing Cisco Data Center Infrastructure)</a></li> <li><a href="https://learningnetwork.cisco.com/community/certifications/ccnp_data_center/dcuci/exam-topics">300-175 DCUCI (Implementing Cisco Data Center Unified Computing)</a></li> <li><a href="https://learningnetwork.cisco.com/community/certifications/ccnp_data_center/dcit/exam-topics">300-180 DCIT (Troubleshooting Cisco Data Center Infrastructure)</a></li> </ul> <p>Similar to the CCNP R&amp;S, I didn’t find the tests unfair, but there is a good deal of useless trivia. This is always going to be the case with mid-level certifications and I wish there was a better way. I spent between 2-3 weeks for each test reviewing my CCIE notes and doing some light labbing, as well as making notes on any new material (thankfully there wasn’t much). I was surprised how much knowledge I’d retained from my CCIE studies for subjects I’m not working with every day. Keeping that infrequently used knowledge has always been a struggle for me, so this process was a useful refresher before I start the process of recertifying my CCIE. I’m going to use the continuing education option to recertify, and it will be with new subjects (programmability) instead of rehashing subjects I’ve already studied. So in the end this was a stressful, but worth while exercise.</p> <p>After completing the testing bonanza, it was time to head to Cisco Live in Orlando. I’m planning on writing a longer recap of Cisco Live in the next few days, so I’ll save the details for that post. One of my tweets did manage to make it on the giant screens at the closing keynote, which was pretty cool.</p> <p class="center"><img src="https://networkbrouhaha.com/resources/2018/07/clustweet.png" alt="" height="75%" width="75%" /></p> <p>Family vacation this year was at Folly Beach, SC. If you have never been, I highly recommend you make the trip. It’s a great beach with lots of fun restaurants, bars and shops. It’s only a short drive from Charleston, which provides plenty to do if you get bored. The highlight was kayaking/paddleboarding down Shem Creek only few feet away from dolphins. We even saw a manatee. Make sure you visit <a href="http://www.jackofcups.com">Jack of Cups</a> and <a href="https://www.tacoboy.net">Taco Boy</a> if you are in Folly!</p> <p class="center"><img src="https://networkbrouhaha.com/resources/2018/07/beachselfie.jpg" alt="" height="75%" width="75%" /></p> <p>It’s almost time to wrap this up. Before I do, I should mention that I have a bit of a role change at work. My title is changing to Principal Engineer, and I’m going to be focusing on product development and research. This aligns well with my interests, and I’m excited to see what the future brings.</p> <p>To finish, here’s a teaser of a few blog posts I’m working on:</p> <ul> <li><a href="/2018/07/CLUS-recap/">Full Cisco Live recap</a></li> <li>CCIE journey/thoughts</li> <li>Guacamole remote access with containers</li> <li>L3 (routing) over vPC</li> </ul> <p class="center"><img src="https://networkbrouhaha.com/resources/2018/02/drink_route_tr.png" alt="" height="25%" width="25%" /></p> Tue, 10 Jul 2018 00:00:00 +0000 http://www.networkbrouhaha.com/2018/07/checking-in/ http://www.networkbrouhaha.com/2018/07/checking-in/ Simple cloud automation with vCD, Terraform, ZeroTier and Slack <p>Earlier this month I had the opportunity to present at the <a href="https://community.vmug.com/communities/localcommunityhome?CommunityKey=84170a74-5453-4c77-abba-74124cd7dd42">Lexington VMUG</a> on cloud automation. I used this opportunity to pull together a few tools I had been experimenting with and combine them for a simple cloud automation example.</p> <h2 id="goal">Goal</h2> <p>My goal with this demo was to deploy a VM in vCloud Director and automate network connectivity via ZeroTier. During testing I decided to include Slack so I could monitor the progress of my scripts. I chose ZeroTier for connectivity because of it’s simplicity. Different cloud providers handle connectivity in different ways. For example, vCD defaults to a “deny any any” firewall ruleset, making it a good fit for ZeroTier. You should follow the best practices of your cloud provider if you’re trying this on your own.</p> <h2 id="tools-used">Tools used</h2> <ul> <li><a href="https://www.vmware.com/products/vcloud-director.html">VMware vCloud Director</a> - VMware’s public cloud solution for service providers in the VMware Cloud Provider Program. We have vCD 9.0 installed in our lab at work, which is where I developed this automation.</li> <li><a href="https://www.terraform.io/">HashiCorp Terraform</a> - An open source tool written in Go, Terraform allows users to define infrastructure as code. Many public cloud <a href="https://www.terraform.io/docs/providers/">providers</a> are supported in Terraform, as well as on prem infrastructure like vSphere.</li> <li><a href="https://www.zerotier.com/">ZeroTier</a> - “ZeroTier delivers the capabilities of VPNs, SDN, and SD-WAN with a single system. Manage all your connected resources across both local and wide area networks as if the whole world is a single data center.” In other words: simple, free<sup id="fnref:1"><a href="#fn:1" class="footnote">1</a></sup>, fast<sup id="fnref:2"><a href="#fn:2" class="footnote">2</a></sup> VPN.</li> <li><a href="https://slack.com">Slack</a> - This cloud based collaboration tool is an easy way to get feedback from cloud-based infrastructure. Slack’s free tier is great for testing simple automation and receiving notifications from your test/dev projects.</li> <li><a href="https://github.com">GitHub</a> - I’m hosting scripts on GitHub, but any web host could fill this need. If you choose another host, you should still use Git for version control.</li> </ul> <h2 id="prerequisites">Prerequisites</h2> <p>Prior to my demo I had installed Terraform and the ZeroTier client on my laptop, created a Linux template in vCD, configured Slack with incoming web-hooks, and uploaded my <a href="https://github.com/shamsway/zerotier-installer">ZeroTier install script</a> to GitHub. My vCD Org VDC is already configured to allow outbound internet traffic. Sanitized versions of my Terraform files are below.</p> <p>vCD Terraform configuration - <code class="highlighter-rouge">labvcd.tf</code>:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>provider "vcd" { user = "[username]" password = "[password]" org = "[org name]" url = "[vCD URL]/api" vdc = "[org VDC name]" allow_unverified_ssl = "true" } </code></pre></div></div> <p>Terraform syntax to clone a linux template - <code class="highlighter-rouge">tf-demo.tf</code>:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>data "template_file" "init" { template = "${file("${path.cwd}/setup.sh")}" vars { ztnetwork = "[ZeroTier Network ID]" ztapi = "[ZeroTier API Key]" slack_webhook_url = "[Slack Webhook URL]" } } resource "vcd_vapp" "tf-demo" { name = "tf-demo" power_on = "false" network_name = "[Org VCD network name]" ip = "allocated" catalog_name = "[Catalog name]" template_name = "[Template name]" memory = 4096 cpus = 1 initscript = "${data.template_file.init.rendered}" } </code></pre></div></div> <p>The <code class="highlighter-rouge">initscript</code> line tells Terraform to parse <code class="highlighter-rouge">setup.sh</code> (below), and sets the contents as the guest customization init script.</p> <p>VM bootstrap script - <code class="highlighter-rouge">setup.sh</code>:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/bin/bash</span> <span class="k">if</span> <span class="o">[</span> x<span class="nv">$1</span> <span class="o">=</span> x<span class="s2">"precustomization"</span> <span class="o">]</span><span class="p">;</span> <span class="k">then </span><span class="nb">echo</span> <span class="s2">"Started doing pre-customization steps..."</span> <span class="nb">echo</span> <span class="s2">"Finished doing pre-customization steps."</span> <span class="k">elif</span> <span class="o">[</span> x<span class="nv">$1</span> <span class="o">=</span> x<span class="s2">"postcustomization"</span> <span class="o">]</span><span class="p">;</span> <span class="k">then </span><span class="nb">echo</span> <span class="s2">"Started doing post-customization steps..."</span> apt-get update <span class="o">&amp;&amp;</span> apt-get upgrade <span class="nt">-y</span> <span class="o">&amp;&amp;</span> apt-get install <span class="nt">-y</span> openssh-server jq <span class="nb">sudo </span>systemctl <span class="nb">enable </span>ssh <span class="nb">export </span><span class="nv">ZTNETWORK</span><span class="o">=</span><span class="k">${</span><span class="nv">ztnetwork</span><span class="k">}</span> <span class="nb">export </span><span class="nv">ZTAPI</span><span class="o">=</span><span class="k">${</span><span class="nv">ztapi</span><span class="k">}</span> <span class="nb">export </span><span class="nv">SLACK_WEBHOOK_URL</span><span class="o">=</span><span class="k">${</span><span class="nv">slack_webhook_url</span><span class="k">}</span> wget https://raw.githubusercontent.com/shamsway/zerotier-installer/master/zerotier-installer.sh chmod +x zerotier-installer.sh <span class="nb">echo</span> <span class="s2">"Installing and configuring ZeroTier"</span> ./zerotier-installer.sh rm zerotier-installer.sh <span class="nb">echo</span> <span class="s2">"Finished doing post-customization steps."</span> <span class="k">fi</span> </code></pre></div></div> <p>This script is run via guest customization when the VM powers on for the first time. It runs apt-get update, installs and enables OpenSSH server, sets some environment variables and downloads the ZeroTier installer from GitHub.</p> <p>Normally I’d put some indentation in the script to make it easier to read, but doing so with guest customization caused the script to break. The <code class="highlighter-rouge">if [ x$1 = x"precustomization" ]; then</code> and <code class="highlighter-rouge">elif [ x$1 = x"postcustomization" ]; then</code> lines are mentioned in VMware documentation as the way to control whether the script is run during pre-customization or post-customization. The <a href="https://pubs.vmware.com/vcd-820/index.jsp?topic=%2Fcom.vmware.vcloud.user.doc%2FGUID-724EB7B5-5C97-4A2F-897F-B27F1D4226C7.html">documentation from VMware</a> uses <code class="highlighter-rouge">==</code> instead of <code class="highlighter-rouge">=</code> for comparison, but this failed in the Ubuntu template I was using. Thankfully somebody <a href="http://markhneedham.com/blog/2012/08/06/vcloud-guest-customization-script-postcustomization-unexpected-operator/">blogged about this</a> error back in 2012.</p> <p>Using Terraform with vCD 9.0 requires using the latest provider code from GitHub, available at https://github.com/terraform-providers/terraform-provider-vcd. Since I am also using the template provider (https://github.com/terraform-providers/terraform-provider-template), I had to download and install that as well.</p> <h2 id="workflow">Workflow</h2> <p>Initialize Terraform. <code class="highlighter-rouge">-plugin-dir</code> instructs Terraform to use the newer provider code that I downloaded.</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ terraform init -plugin-dir /usr/local/go/bin/ </code></pre></div></div> <p>Run Terraform. <code class="highlighter-rouge">parallelism=1</code> instructs Terraform to run one task at time. By default, it will run tasks simultaneously. This restriction is covered in the <a href="https://www.terraform.io/docs/providers/vcd/r/vapp_vm.html">vCD Terraform provider docs</a></p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ terraform apply -parallelism=1 data.template_file.init: Refreshing state... An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: + vcd_vapp.tf-demo id: &lt;computed&gt; catalog_name: "MattLEX" cpus: "1" href: &lt;computed&gt; initscript: "[snip]" ip: "allocated" memory: "4096" name: "tf-demo" network_name: "MattLEX" power_on: "false" template_name: "ubuntu_16.04_small" Plan: 1 to add, 0 to change, 0 to destroy. Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: yes vcd_vapp.tf-demo: Creating... catalog_name: "" =&gt; "MattLEX" cpus: "" =&gt; "1" href: "" =&gt; "&lt;computed&gt;" initscript: "" =&gt; "[snip]" ip: "" =&gt; "allocated" memory: "" =&gt; "4096" name: "" =&gt; "tf-demo" network_name: "" =&gt; "MattLEX" power_on: "" =&gt; "false" template_name: "" =&gt; "ubuntu_16.04_small" vcd_vapp.tf-demo: Still creating... (10s elapsed) [snip] vcd_vapp.tf-demo: Still creating... (7m30s elapsed) vcd_vapp.tf-demo: Creation complete after 7m32s (ID: tf-demo) </code></pre></div></div> <p>vCloud Director clones the template based on my Terraform config. I instruct Terraform to not power on the VM after creation. This is due to a bug in the vCD Terraform provider that tries to apply guest customization after power on if done during cloning. This is not ideal and could be automated via an API call to vCD, but I’m manually powering on the VM.</p> <p>When the VM boots, guest customization runs the init script (guest customization scripts are limited to 1500 characters, so keep that in mind). The init script downloads a ZeroTier <a href="https://github.com/shamsway/zerotier-installer/blob/master/zerotier-installer.sh">installer script</a> from GitHub. It is based on the official ZeroTier linux install script at http://intstall.zerotier.com/, but pared down to work on Ubuntu 16.04 only. I’ve also modified it automatically authorize the new VM via ZeroTier API, and send feedback to Slack.</p> <p>As the script runs it posts messages in Slack. The final message displays the ZeroTier IP assigned to the VM.</p> <p><img src="https://networkbrouhaha.com/resources/2018/03/vcd_slack.png" alt="" /></p> <p>Once I have the ZeroTier IP, I can SSH to the VM to continue whatever setup I would normally complete.</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ ssh user@10.244.80.210 Welcome to Ubuntu 16.04.4 LTS (GNU/Linux 4.4.0-116-generic x86_64) * Documentation: https://help.ubuntu.com * Management: https://landscape.canonical.com * Support: https://ubuntu.com/advantage 2 packages can be updated. 0 updates are security updates. Last login: Fri Feb 23 17:27:57 2018 from 192.168.0.4 user@tf-demo:~$ ping www.google.com PING www.google.com (172.217.6.4) 56(84) bytes of data. 64 bytes from ord38s01-in-f4.1e100.net (172.217.6.4): icmp_seq=1 ttl=47 time=19.4 ms 64 bytes from ord38s01-in-f4.1e100.net (172.217.6.4): icmp_seq=2 ttl=47 time=19.4 ms </code></pre></div></div> <p>When I’m finished with the VM, I can destroy it with Terraform. Note that this does not de-authorize the VM in ZeroTier, which is something you would want to do if you were using this in production or on a frequent basis.</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ terraform destroy -parallelism=1 data.template_file.init: Refreshing state... vcd_vapp.tf-demo: Refreshing state... (ID: tf-demo) An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: - destroy Terraform will perform the following actions: - vcd_vapp.tf-demo Plan: 0 to add, 0 to change, 1 to destroy. Do you really want to destroy? Terraform will destroy all your managed infrastructure, as shown above. There is no undo. Only 'yes' will be accepted to confirm. Enter a value: yes vcd_vapp.tf-demo: Destroying... (ID: tf-demo) vcd_vapp.tf-demo: Still destroying... (ID: tf-demo, 10s elapsed) vcd_vapp.tf-demo: Destruction complete after 19s Destroy complete! Resources: 1 destroyed. </code></pre></div></div> <h2 id="lessons-learned">Lessons Learned</h2> <p>vCD support in Terraform is not great, but appears to be getting better. The current Terraform <a href="https://github.com/terraform-providers/terraform-provider-vcd">vCD provider</a> is based on an older vCD/vCloud Air <a href="https://github.com/UKCloud/govcloudair">Go library</a> developed for vCD 5.5. There is a <a href="https://github.com/terraform-providers/terraform-provider-vcd/blob/master/v2Plan.md">plan</a> to develop better support for newer releases of vCD. VMware has also <a href="https://github.com/vmware/terraform-provider-vcloud-director">released their own vCD Terraform Provider</a>, but it is not clear if this will be included in future Terraform releases. One interesting note is the new provider does away with the native Go library in favor of using gRPC + <a href="https://github.com/vmware/pyvcloud">pyvcloud</a>. I am participating in a VMware Cloud Provider Technical Advisory Board meeting next month and I will attempt to get some clarification on the future of vCD with Terraform.</p> <p>Looking back on this exercise it is clear that there are many hoops to jump through to automate vCD and Terraform. vCD is not <em>widely</em> deployed, but it is used at some of the “second tier” and smaller cloud providers. Over the last two years VMware has recommitted to improving vCD, with the 9.0 and 9.1 releases providing huge improvements to the platform. I believe vCD will continue to improve, as well as gain traction and visibility. On the other hand, there are many other public cloud options, and all of them have robust support in Terraform. Anyone can take the Terraform configuration I wrote for vCD and adapt it to another provider with minimal effort.</p> <h2 id="final-thoughts">Final Thoughts</h2> <ul> <li>Think about networking connectivity to the cloud - have a plan. Public/HTTPS, IPSec VPN, SSL VPN, ZeroTier, SD-WAN and Direct Connect are all feasible options.</li> <li>Find a cloud provider you can use for practice - AWS, Digital Ocean, Azure, or vCD.</li> <li>Practice building infrastructure with Terraform or similar tool.</li> <li>Use git version control for your scripts.</li> <li>Remember: <em>Don’t put your passwords and API keys in your public github repo</em>!</li> </ul> <div class="footnotes"> <ol> <li id="fn:1"> <p>GPL license / Up to 100 devices / Requires license to embed in commercial products. <a href="#fnref:1" class="reversefootnote">&#8617;</a></p> </li> <li id="fn:2"> <p>Quick setup, but actual traffic may proxy through ZeroTier servers. There is no throughput guarantee. <a href="#fnref:2" class="reversefootnote">&#8617;</a></p> </li> </ol> </div> Sat, 24 Mar 2018 00:00:00 +0000 http://www.networkbrouhaha.com/2018/03/vcd-terraform-example/ http://www.networkbrouhaha.com/2018/03/vcd-terraform-example/ NSX automation example <p>I’d intended to include this in my <a href="/2018/03/network-automation-book-review/">review of Network Programability and Automation</a>, but I forgot, and it was long enough anyways.</p> <p>Over the past few months I’ve had some conversations with co-workers about backing up configurations from Edge Gateways in vCloud Director. After reading through the first few chapters of the aforementioned book, I decided to put the pieces together and give it a shot. There was some additional Googling required to write this, but a good portion of it comes directly from info I learned from Network Programability and Automation.</p> <p>You can check it out here: https://github.com/shamsway/vcd-edge-backup</p> Wed, 07 Mar 2018 00:00:00 +0000 http://www.networkbrouhaha.com/2018/03/NSX-automation-example/ http://www.networkbrouhaha.com/2018/03/NSX-automation-example/ Book Review: Network Programability and Automation

I was anxious to get my hands on Network Programability and Automation by Jason Edelman, Scott Lowe, and Matt Oswalt as soon as I saw the first announcement that it was being written. Network automation is a subject that I am deeply interested in, so I was excited when the early access edition showed up on Safari Books Online. I was giving a lot of thought to starting this blog when I began reading, and by the time I made it through a few chapters I knew that I wanted a review of this book to be the first “real” post on my blog.

Disclaimer 1: This review is based off an early access edition of Network Programability and Automation on Safari Books Online. The final version is not available on Safari yet, but based on the Table of Contents at the O’Reilly site for the book, there were some structural differences between the early access and final editions. I have very few criticisms of this book, but I’ll do my best to update this review once I get my hands on the final copy.

Disclaimer 2: I’ve never written a book review before. I am somewhat writing this for writing’s sake. My goal is to build up some writing chops as I ramp up this blog.

I absolutely love this book. If you’re a network engineer with any interest in where the industry is going, it is a must read. Automation is a scary topic for a lot of engineers due to the perception that you have to be a programming wizard to do it. Network Programability and Automation effectively dispels that myth by providing an easy to follow blueprint, and it doesn’t presume any previous programming experience. This book is a fantastic resource for both novices and seasoned programmers. It’s worth noting that this book focuses on open source tools like Ansible, Salt and StackStorm. Some of these tools have commercial variants, but if that’s not your cup of tea, there are other closed source options on the market. No matter what tool you choose for Network Automation, this book is an amazing resource. For the remainder of this review I’m going to go chapter by chapter to give a taste of what the book covers, as well as some additional useful resources.

Chapter 1 begins with an overview of current network industry trends: Software Defined Networking, and everything that comes with it. The authors are careful to avoid providing a formal definition of SDN, which is a smart choice. You can ask ten different vendors or engineers what SDN is, and get ten different answers. They do, however, nail down the ecosystem that is driving this software defined trend. The release of the OpenFlow protocol is really the point where SDN started, so the book begins with a brief history and overview of OpenFlow. NFV, VXLAN, virtual switching, APIs, fabrics and whitebox switching are touched on as well. This is a good primer for the engineer that hasn’t been keeping up with all of the recent industry hype and hand-waving.

Chapter 2 provides a high level over view of the what, why and how of network automation. This chapter spells out the benefits of network automation, and typical use cases are covered. We are seeing a shift in the industry away from SNMP and CLIs to APIs and NETCONF, and this is covered as well. It’s not mentioned in the book, but VMware NSX is a great example of this. There is no CLI, only API or GUI access for configuration. There is no SNMP polling either - the NSX API is the only way to monitor the environment. Of course, VMware provides commercial tools to monitor NSX, but it’s certainly feasible to roll your own monitoring solution once you’re familiar with interacting via API.

Chapter 3 delves in to Linux history, basics, and networking. The early access edition had both a Linux chapter, and an appendix devoted to advanced networking in Linux. I’m assuming that the appendix was rolled into this chapter, which makes perfect sense.

Linux has become the de facto language of the data center, and it is increasingly important that network engineers have an understanding of it. There is a long list of open source projects used for network automation, so the ability to navigate a linux server is table stakes for getting started. The caveat is you could use a commercial network automation product that requires no Linux knowledge, but what fun is that? This chapter is a crucial resource for engineers with no prior Linux experience, and a good refresher for veterans of the operating system.

The early access edition discusses VRFs and Namespaces on Linux in the “Advanced networking in Linux” appendix. At the time of writing, VRF support in Linux was not complete, so the authors focused on Namespaces to provide a VRF-like experience. VRFs are now fully supported on Linux. The folks at Cumulus Linux have great writeup on this topic.

Chapter 4 is a Python primer. If Linux is the language of the data center, Python is the language of open source network automation. This chapter explains programming concepts that are foundational for automation – data types, loops, file I/O, and APIs. This chapter goes deep enough into Python to get you started, but there are plenty of additional resources available to further your Python knowledge. The oft-repeated question, “Is coding a requirement for Network Engineers now?” is addressed as well. I’ll let you read the book to see how this is answered, and I agree with the answer given in the book.

If you want to learn more, these are my recommendations.

The early access edition of this book included an appendix on NAPALM (Network Automation and Programmability Abstraction Layer with Multivendor support), a Python library for interacting with network hardware. I do not know if this content was moved to the Python chapter or not, but the NAPALM is mentioned several times in this book and it’s widely used by other tools. It’s worth your time to read the docs, follow their blog, and star their GitHub project. It’s safe to say that when you’re starting out with network automation, you’re going to use NAPALM in some way, shape or form.

Chapter 5 explains data models like JSON, XML, YAML, and YANG. The data models are equated to programming elements introduced in the previous chapter. This is not the most riveting information in the book, but it becomes very important as it is foundational to the tools introduced later on, as well as when consuming APIs. I wish there was a little more information about YANG in this chapter, but there are plenty of other resources out there. Once you’ve read through this book, you can find some great YANG examples at these links:

  • https://github.com/YangModels/yang
  • https://github.com/openconfig/public

Chapter 6 is similar to the previous chapter. It introduces the Jinja templating language, which is what you will use to build your configuration templates. This chapter is full of plenty of great examples and advice. Jinja was one thing I had very little understanding of before reading this book, and this flipped on the light switch for me. I believe all of the network automation tools introduced later in the book use Jinja, so it is an important concept to understand.

Chapter 7 goes into great detail on APIs, both RESTful and non-RESTful, and NETCONF. My takeaway from this chapter that NETCONF is more complicated than the other options. Part of this is from having to deal with XML, and more widespread adoption of RESTCONF should ease this a bit. There is a great overview of using the Python requests library for interacting with APIs. There are also useful examples using Cisco Nexus NX-API, IOS-XE RESTCONF, Arista EAPI, NETCONF via ncclient, and an intro to the Python netmiko library.

Chapter 8 is a short detour before getting to the real meat of this book. This chapter explains git version control history, terminology and use cases along with several working examples. Understanding git is an important skill since many existing network automation tools and templates are hosted on GitHub. Any new templates or scripts you create should definitely be version controlled with git, whether that is via GitHub or a private git solution.

Chapter 9 is a lengthy chapter on automation tools. It starts with an architecture review of existing tools: agent-based/agentless, centralized/decentralized, open or proprietary protocol, etc. The rest of the chapter is devoted to the tools themselves.

The first tool examined is Ansible, which is the one I was least familiar with and most interested in. All of the basics are covered: inventory files and groups, variables, playbooks, using Jinja templates, gathering device data, generating config files and pushing configuration to devices. After reading through this session my mind was full of potential use cases, and I felt it provided a strong enough foundation to get me started. It is worth noting that there are some syntax changes in Ansible 2.5 to consider if you are planning on using that version. Ansible Tower is the commercial offering for Ansible, which provides a GUI workflow designer and other features. AWX is the open-source/bleeding edge version of Tower, and is worth investigating if you’re considering Ansible.

Salt is covered next, and while I was a bit more familiar with this tool, I’d never used it for network automation. Again, the authors do a great job of covering the basics along with providing some examples (you’re probably noticing a theme here.) I learned that Salt is much more complex than I’d originally realized. It’s completely capable of complex automation use cases, but it does require more thought and planning to effectively use. Some engineers from CloudFlare have published a free eBook that goes into greater detail on using Salt for Network Automation: Network Automation at Scale

StackStorm is covered last. This tool is primarily event driven - it’s described as the “If This Then That” of IT infrastructure. My impression is that it could be set up to work like Ansible or Salt, but that is not really its intended use. There are many moving pieces to understand: Actions, Workflows, Sensors, Triggers, Rules, Packs and Workflows. The event driven nature is a good fit for automated remediation, or chatops via Slack and Hubot. StackStorm has native plugins for these tools.

Here’s my take:

  • Ansible: Not too difficult to get started with - appropriate for simple and complex scenarios.
  • Salt: Feature rich but likely takes some planning to effectively use. It is a good option if you want to automatically respond to events.
  • StackStorm: More than just an automation tool, StackStorm is a system you could use to drive a complex IT automation and chatops environment.

This chapter was jam packed with goodness. If I could make one small criticism, it is that I wish this had been split up into three separate chapters. So. Much. Content.

We’re almost done now, I promise :grin:

Chapter 10 is an intro to Continuous Integration/Continuous Development (CI/CD). Typical CI/CD tools and workflows are covered, as well as a basic example. The authors make a point to stress the importance of testing your automation with a CI/CD tool if possible. I do wish there were a few more examples in this chapter. This is probably the hardest nut to crack for anyone starting from scratch with Network Automation. I’m going to continue digging into this topic, so hopefully there will be some future blog posts on the subject. Thankfully we’re starting to see Docker-based images from many networking vendors, and that will help streamline the CI/CD process.

Chapter 11 discusses building a culture for Network Automation. Apart from Chapter 9, this is the most important chapter in the book. Knowing how to automate is great, but it’s not much help if there is resistance to change in your organization. This chapter covers how to get started in a “traditional” (i.e. old-school) IT organization, the importance of getting executive buy-in, and how to navigate the build vs. buy decision. Some of the best advice in the chapter is around dealing with and embracing failure. No book on automation is complete without addressing the “Will automation take my job?” question, and it is handled beautifully.

Closing thoughts

There is nothing in this book that is going to increase your understanding of OSPF or BGP. What it will do is give you the tool set to effectively monitor, configure, and remediate your network. This book covers so much information, yet there is still so much more to learn. I’m going to wrap up this incredibly long post with a link to this video from NANOG 72 titled “Network Automation: Do I Need Expensive Tools To Do Meaningful Automation?” In it, Patrick Moore gives some great advice for getting on the “automation on-ramp” and the presentation is a great follow up to Network Programability and Automation.

In the words of Jason Edelman, Scott Lowe, and Matt Oswalt: be the “automator”, not the “automated”.

Edit: I’d intended to include an example script I’d written after reading this book. Check it out here.

Tue, 06 Mar 2018 00:00:00 +0000 http://www.networkbrouhaha.com/2018/03/network-automation-book-review/ http://www.networkbrouhaha.com/2018/03/network-automation-book-review/
Hello World <p>Let’s get this straight first: I’m from Kentucky, and we say y’all. I had to get that out of the way.</p> <p>Hey y’all. Every blog needs a first post, so here it is. If you really want to get to know my history, you can check out my <a href="/about/">about</a> page. Here’s the tl;dr - I’m a Network Architect for a regional MSP/CSP/VAR in Lexington, KY. I earned my CCIE (Data Center) in March of 2017, and I’ve spent the last twelve months doing anything besides studying. Unfortunately it’s time to get serious again. Over my career I’ve drawn a lot of inspiration from <a href="/links/">bloggers and podcasters</a> in the networking community, and I’m quite sure I would not have gotten to where I am without their examples. While this <a href="http://blog.ipspace.net/2018/01/how-to-become-better-networking-engineer.html">post</a> from Ivan Pepelnjak was not the reason I decided to start this blog, the items he lists are a good summary of why I’m doing it.</p> <blockquote> <ul> <li>Mentor younger engineers;</li> <li>Educate non-networking professionals how networking really works;</li> <li>Blog about your experiences (but make sure you’re technically accurate);</li> <li>Write documentation for open-source software;</li> <li>Create or contribute to open-source software (it can be as little as one-line bugfix);</li> </ul> </blockquote> <p>I can’t say that i’m going to be able to check off every item on this list, but I’m going to give it a shot. Along those lines, I want to put some other goals in writing.</p> <ul> <li>Write at least one blog post per month</li> <li>Participate in my local and regional tech communities (e.g. VMUG, NOGs, Podcasts)</li> <li>Work on obtaining my VCP6-NV certification</li> <li>Work on obtaining my AWS Certified Solutions Architect - Associate certification</li> <li>Write some Python, ideally to automate NSX</li> <li>Become proficient with Ansible</li> </ul> <p>I’ll be blogging about those things over the coming year, as well as anything else that catches my interest. Until next time - stay thirsty, my friends.</p> <p class="center"><img src="https://networkbrouhaha.com/resources/2018/02/drink_route_tr.png" alt="" height="25%" width="25%" /></p> Tue, 27 Feb 2018 00:00:00 +0000 http://www.networkbrouhaha.com/2018/02/hello-world/ http://www.networkbrouhaha.com/2018/02/hello-world/