TOR Search Engine Using Apache Nutch

A local educational institution requested assistance with their TOR web crawler project. The objective was to create a database of malicious files found through visiting websites on The Onion Router network. The software used for this project was Apache Nutch, Elasticsearch, and Kibana. The overall problem was that Nutch wouldn’t send crawled data to its specified location, and the team would have to find a method for indexing this information properly. The team would also have to modify configurations to index specific file extensions. The end result would be a dedicated server consistently scanning TOR pages to form a large search engine.

Project Students: John Bovey

Testimonials

The Living Lab is like having an IT job in the field. You are pushed and pulled in many different directions with many different projects. You are thrown into situations that need to be figured out. The guidance is there, but nothing beats that feeling of figuring it out by yourself. Aaron Hook

Before Living Lab, I had no work experience related to my IT career. I am grateful to my teachers for their support and encouragement on my projects. I experienced projects hands-on and improved my soft and technical skills. I learned how to work independently and also with a team as needed. I recommend students learn by direct practice and experience the Living Lab. It will introduce you to your prospective career! Ibrahim Issakha Doude

The Living Lab has been an enjoyable experience. It has provided new skills while strengthening old ones and has given me insight into the technical workplace. Wendell Matthew Klene

The Living Lab has been a great experience for me throughout the semester. I worked with some technologies that were previously unfamiliar and consulted with a client to develop a good solution to implement for them. Working in many unfamiliar and uncomfortable situations truly pushes you, and I believe it’s the best way to foster growth in any profession. So for the Living Lab to give that opportunity is a great thing. Nathanial William Adams

I have definitely determined that this class could be just like a real life IT position. You never know how long a project might take, even a simple task. I’ve had some tasks take only a couple of hours, and I worked on one project for 36 hours straight. But I feel that Living Lab is helping to prepare me for what will come in the professional world. Eugene Gephart

Throughout this class, I have learned a great deal of information. It gave me an idea of how how the IT world runs and prepared us for it. Abdulrahman Jassem M. Alzaman

The Living Lab was a great opportunity to gain different technical experience in the IT field. The projects were all challenging and simulated a real-world environment to deliver upon. Nick Barnett

The Living Lab has been very educational for me this semester. I gained experience in collaborating with a client in order to install a Polycom Videoconferencing system, to provide a new way of communication for the client. Paul D. Bailey

The Living Lab experience has helped me better understand my abilities and what I need to work on. I worked with others in a team and also learned to lead others when my project needed help. In a previous IT job, I did similar tasks to those in the Living Lab, and it shows just how good an experience this can be to help others be better prepared when they graduate. Justin Tanner Andrew

The Living Lab experience has helped me see what it can be like in the real world. David C. Bryant