Week 20 – Project 1 Part 2

Weekly Learning

This week we worked on the exam. One of the interesting things about the exam is that despite it being open book, it is still a challenge. I’ve been very careful to craft each query and double check against each data point to make sure I am getting the answer I believe it best fits the question, but some of the queries are very complex and make me second guess myself.

We have also been working on the OLAP part of the project. I have to say that I had never enjoyed database work until now. Before, most of my queries were just basic selects, and getting data was more of a chore. Having practice more both with the designed, and how data is composed, I have started to enjoy writing queries that extract data. Where JOINS were a total mystery for me before, they are now almost second nature. I have been doing a lot of playing around with JOINS, GROUP BY, subqueries of virtual tables, and other fun features of myself.

The OLAP assignment has been a great opportunity to also start shifting my thinking into what data should be collected and processed. I realized because this is all fake data, our questions are probably naive. However, I have a good opportunity to compare it against the questions that my coworkers ask, or the assumptions they make when they request new features. Looking at how I would organize the data, and trying to understand what of my data is valuable is a very important step.

I guess I can only compare it to how people say that half of being good at a job such as programing, is knowing how to google. In this case, I’m getting much better at knowing what questions need to be asked, and how my data can either answer it, or what would need to change to make sure it can answer those questions.

Overall, I can honestly say I came to the class expecting it to be one of the classes that you dread to take, and push through, to being one of the classes I’ve most enjoyed.

Posted in CST-363, CSUMB | Comments Off on Week 20 – Project 1 Part 2

Week 19 – Project 1 – Database Design

Weekly Learning

This has been a very demanding week. Project one has been a challenge with both the creation of the database, as well as creating an application to properly visualize the data. I think we started working on it a little later than we should and that meant we had to pull some longer hours getting everything ready. However, I am pretty satisfied with the result with how little time we actually had to accomplish it.

Victor and I decided to go with a game database. This is fairly unconventional since it does not necessarily have as much transactional data as other application would. But, I think it presented an interesting challenge for both of us. Game have a lot of very interesting Business Intelligence data working behind them, as players move through different levels, or value one item over another when making a purchase. Understanding what your users do is important for any business, and games are no different. This data is a bit more abstract than a reservation system would be, since it must be derived from activity rather than numbers, but it’s equally important to break down those behaviors and make sense of them so the operators of the game can make better decisions that will capitalize and monetize the users.

I also spent a bit of time doing research on OLAP and data cubes. I have worked in the past with BI. But this is the first time I have actually sat down to understand how this data is constructed, processed and broken down to be useful on the backend. In the past I have just been a consumer of the data, and that was good enough to get me through the day, but as I have been digging through the topic, I have found it to be very interesting. The analogy of the hot kitchen is incredibly apt with some of the requests I have gotten in the past. Building the datasets was an ugly process but generated reports that could be very useful to some of my customers.

Posted in CST-363, CSUMB | Comments Off on Week 19 – Project 1 – Database Design

Week 18 – Database Design

Weekly Learning

Database Design is a very interesting topic. My personal experience designing databases has been very limited professionally. Normally, I have gone by what the data looks like and designing around it. Comparing this to how the chapter describes database design, I think the overall outcomes have not been very far from what is expected, but data but this has been certainly based on good tribal knowledge and mentorship. Having read and worked on the cases for normalization, I think I have a much better understanding of why those designs worked. More importantly, I understand why normalization is so important, and why we often sacrifice it to make data more accessible.

We’re also starting to deep our toes into Data Science and how sql supports aggregation. Since I am very much interested in the monetization and targeting strategies, this is a topic that is very dear to me. I often have to look for ways to make sense of purchase trends and patterns, and look for ways to visualize that data so users can look into it. In the past, I’ve used code to aggregate and process the data, or used specialized tools like Druid. When data gets to my hands, it’s often already gone through the ETL process, so i’m very much looking forward to the next couple of weeks to better understand the whole process. But using GROUP BY and other aggregators, already helps me!

Posted in CST-363, CSUMB | Comments Off on Week 18 – Database Design

Week 17 – Building Queries on MySQL

Weekly Learnings

This is the first week coming back from the winter break, and we’re jumping right into the action. First, I think the book we’re working with is fantastic! I love the simple, elegant, and pragmatic approach of only using the right side for examples and as reference. It makes both working through the exercises and understanding the logic behind it much easier.

I have been fortunate enough to use MySQL at work, but normally development is done through APIs, and actual access to the database is limited to some select statements to verify data. This week, was particularly satisfying as I finally got to get a solid grasp on both how JOIN statements work. This has been a question that I have previously gotten during interviews, and felt I have come short explaining properly because the concepts were not clear.

I also found this blog on the subject to be very good. It has a better way to graphically represent the INNER, and OUTER join relationships. I have to agree with the author that the ven diagram representation of this functions is just not adequate.

Say NO to Venn Diagrams When Explaining JOINs

Overall, i’m very much looking forward to this class! I think learning more about databases is fundamental for any developer!

Posted in CST-363, CSUMB | Comments Off on Week 17 – Building Queries on MySQL

Week 15 – Media Final

This has been a really good class. My only regret is that we didn’t dive deep enough into media manipulation. I would have liked to spend more time doing sound manipulation, and learning to mixers work. But I realize this is a short class, and we had too many topics to cover. Demystifying images and filters, and creating our own was very interesting, and it puts in perspective why and how applications like Snapchat are so successful. It also means we have the tools to start our work into something similar to this.

This last lab was also very interesting. I do love parsing strings, and putting a small page together was very interesting. It’s clear this was just to touch on how HTML works, because python already has very robust HTML frameworks that can make a lot of this work trivial. But as an experiment on delivering a simple webpage, it was pretty interesting. I think for me, more than the HTML, I really enjoyed finding a creative way to show the word count. I did research into the heatmap algorithm, and then curving the count so there is a better color distribution was the most fun.

The final project was a bit more rough. We spent a lot of time working on it, and trying to deliver the best user experience as well as a complete product. The biggest raw improvement was using smaller images, which translates to a lot less work copying pixels from one picture to another. Then optimizing so we only draw items we care about and not rendering maps on every action. Those small tweaks made a huge difference in performance, and overall, helped us deliver a better product.

Below is the demo video for our game:

 

What I would most recommend students taking this class is to come with some basic knowledge of python. Even a light pass on basic syntax and data structures can be incredibly helpful. It’s also very valuable to brush up on all your array, and dictionary knowledge. Those are very important data structures that will be used constantly through your labs and knowing how they work will make your life much easier. Finally, be very curious and be willing to experiment with the knowledge. Some of the most fun I had in this class was when I took the assignments off the rails and just tried to see what happened when I changed images, or sounds to do other things.

Being curious in this class has big payoffs in the way of knowledge and understanding.

Posted in CST-205, CSUMB | Comments Off on Week 15 – Media Final