Hello, and welcome back to this course in which we're talking about reconnaissance in initial access. In this video, we're going to be focusing on the reconnaissance side of this, talking about Shodan and how we can use it with Python. If you're not familiar, Shodan, as we see on the screen here, advertises itself as the search engine for free Internet of everything. It's essentially the equivalent of Google if you want to look up IoT devices and other Internet-connected systems. During the reconnaissance phase of a cyber security engagement, Shodan can be invaluable for collecting information from open source information repositories. Shodan, as you see here, has a search bar just like Google. Let's see what it brings up for Google. We can either type in the phrase google here, just like a keyword search, or we can refine it a little bit more using Shodan search filters. If we say org:Google LLC, we're going to get systems that belong to Google LLC. We had search there, give it a moment to run and we get some results here. In total it has brought up about 79 results, and we see here that they are associated with Google LLC. We could dive into one of these results. Let's take a look at this one. This shows the information that Shodan can provide for a particular IP address. We've got the IP. We know apparently it's a Cloud-based honeypot, a few open ports. We see 443 for HTTPS, we see 8080, etc. We have a bit of general information about it, the banner grabbing and other information about each of the ports that's open. Here we get information about the web server, its SSL certificate. Scrolling down 502, we get the results when Shodan tries to query that port, 8080, we see again another web server, and then 8880, we see again another web server. The value of Shodan for us and why we're taking a look at this site before we dive into some Python code is that it provides a lot of the information that we need in reconnaissance, and we don't have to get it ourselves. If you're performing an ethical hacking engagement and you want to know which ports are open on IP address 34.66.97.189, you can either use Nmap to try to scan those, and that's scan's going to show up, or you look it up in Shodan and here's your listing right here of the available ports. One of those fairly active, fairly detectable, the other, quite stealthy, and that's not even taking into account the potential to discover that this particular IP addresses of interest using Shodan. We didn't start out looking for 34.66.97.189, we looked at things that are associated with Google. If you're performing an ethical hacking engagement against another organization, looking for the ones that are known to be affiliated with Bare, it can provide an initial collection of systems that you can start digging into and diving into. I pulled up Shodan here in the web browser because it's a little bit easier to look at here. However, this is a course about Python, so we're going to be talking about how to use Shodan with Python via the Python API. Shodan has a developer section here, and its got Python libraries. It's very easy to access the information in Shodan via its API. If we go down and click on the raw data section here, instead of regular view, we see the information that we'd get if we query in this particular IP address using Shodan and the API. If we scroll down, we see we've got a data section here and then we start getting information about it. We have a provider, Google, we have HTTP from one of the ports that's been visited, host names, IP addresses, ISPs, etc. There's a lot of valuable information in here for us and we're going to talk about extracting that information in Python. For that, we'll minimize this and go to our Python code. Our Python program in this video is going to be called ShodanSearch.py because we're going to be using it to search Shodan. As I mentioned, Shodan includes a developer pin with libraries for multiple different programming languages. One of these is Python, and so by installing Shodan and importing it with import Shodan, we have access to the Shodan API in our Python code. The Shodan API is only available if you are a paid member of Shodan, it's a onetime fee and gives you access to the API with a certain number of API credits, per interval. In order to actually use the Shodan API, I need to load in my Shodan API key, and then I can create an instance of Shodan with shodan.shodan of key, and we're going to call that API. With the API setup, we can start using Shodan for reconnaissance. In reconnaissance, we have two different goals we might be wanting to achieve. We might want to go broad in reconnaissance. In our initial mapping of the attack surface, we need to know every single system that's associated with a particular organization or that meets certain criteria, such as running a particular type of service. We'll be doing that in our first Python function here that we call queryShodan. The other goal of reconnaissance might be to go deep and gather information about a particular host. Say for example, we've determined that a particular IP address is of interest, and so we just want to ask Shodan about what is at that particular IP address, and that's going to be on our Shodan lookup file, or function. Let's start with queryShodan. Our goal here is to do something very similar to what we just did with the Shodan website. We just typed in org Google LLC and got a list of the different IP addresses and set Internet connected devices that are associated with Google. We can send that same query in and get what should be an identical set of results. Within that set of results, we've a certain number of matches, and in each of those matches, we want to be able to extract certain information that's going to be useful to us later on. We're going to say the IP address of the results. We want to know the port numbers associated with it, and that'll give us clues about what services might be running on that system, etc. That's the goal of queryShodan here, is just give us a high level view of our target attack surface. I'm going to import these functions into our Python terminal here, and then you can try calling them. From ShodanSearch import star, is going to set up my API and give us access to these two functions. Now if I say, results equals queryShodan and then use that exact same query that we used on the website, we should get a set of results. I forgot my closing quotes there, give it a moment to run, and then if we do length of results, we'll see we got 74 hits. Let's take a look at results of zero. Print out results and the dictionary, it can't query results of zero, it would be a list. Here we see IP addresses with associated port numbers. This provides us with a high level view of what's going on in Google's network, because we've got all of these IP addresses that supposedly belong to Google LLC. With this information, we can start trying to build a network map about what might be going on on these systems. For example, here, we see 35.239.237.195 with port 25 open. That implies that we've got an SMTP server running on that particular system. On other systems, we have, say, port 8880, port 80, etc. For example, port 80 is a web server. Looks like we've got a port 22 here, 443 here, etc. Now we've got a broad view of some of the results associated with Google LLC. Let's dive a little bit deeper into one of them. For that, we'll use ShodanLookup with a particular IP address. In this case, instead of the search command that we used with the Shodan API, we're going to be using host and then we just pass it the IP string. Let me grab this first IP address here to use for our query. In this one, we're going to get a lot of the same information as we would here, but we're going to grab a few more fields if possible. We definitely want to know the IP address, which we already have, and any port information we can get for that particular system. However, we also want to go a little bit further and try to learn a bit more about what service is running at that port. If you're familiar with Nmap and how it does service identification, Shodan can do that a little bit as well in some cases. The information that it provides can come in a couple of different forms. If possible, the product and version are the best values we can get for a service. That'll tell us exactly what service is running and its version number. That's the information you can use to look up CVEs and learn about potential vulnerabilities. The CPE is also good. That's just a standardized representation of that product and version number. We can parse that to get the information we need. But all of these are optional fields. The other thing we're going to grab is what we're going to call the banner, which is stored in the data field here. This is the response that Shodan received when it queried that particular service. This could be an HTTP response or it could be an actual banner from a service like SMTP, FTP, etc. By grabbing all of this information, we're trying to maximize our chances of actually identifying the services running at this IP address. Now, let's give it a try. In here, we'll do results equals ShodanLookup. We're just going to use this first IP address here. Close that out, and let it run. Results, we'll print it out first. We see we've got a fair amount of information here. Let's look at each of them in order. Results of zero, we see we've got that port 25 that we saw there associated with this IP address up above here, and we get the banner. Looking at this banner, we've confirmed that it's an SMTP server, which we already could have guessed from the port number, but not much else. This particular banner isn't really telling us that it's, say, VSFTPD or something like that, which would let us start to work on identifying potential vulnerabilities. Looking at our next results, we get a little bit more. This is port 82, a bit of an unusual port here. However, if we look at our banner, off the bat, we know we've got some webserver running here because we've got an HTTP 200 response. Diving further in, we see that we actually got a few of our optional fields here, which is great because this is telling us that it's a Microsoft IIS HTTPD, so HTTP server running Microsoft IIS version 10. That's the exact information that we're going to need to start looking for CVEs here. Also, we've got jQuery, Bootstrap, etc, more information about the system that we can start looking for potentially exploitable vulnerabilities. Demoing a couple of different uses for Shodan here with Python. We're going to be tying these into our process of information gathering once we're done covering all the various bits and parts in the following videos. Thank you.
