Develpreneur: Become a Better Developer and Entrepreneur

One of the architectural challenges that are often overlooked is large file storage. These may be documents, images, or other general binary structures. This situation is also becoming a much more common challenge for applications. We often have a feature for storing a profile picture for users, PDF documents, or other similar features. This area is something worth spending time on during the creation of the software architecture. Storage and Retrieval Any time you store data, you have to include the mechanism to retrieve it. In the world of data stores, that often means a way to index metadata or maybe the content. When we look at large file storage, we have several options. There is the old solution of file system storage with a path, or we can look at blobs, cloud storage, and other solutions. Each of these options has pros and cons to consider as we architect our solution. We also need to think about how to retrieve and restore that data to its original form. This step may be a simple task, or we might have encryption, compression, and file format steps to deal with. For example, we can store a resume as a file. However, we may accept Acrobat, Word, and other formats. We are usually not going to be able to maintain the original name when we store it to a file system. However, does that include an extension as well? Moving Data One of the essential factors in how you architect a solution to this problem is how often you will need to move around data. If you will only have one server and can support the needed space, then you might find a file system easier. When you move to multiple servers, then a cloud or SAN will make more sense. When you need to keep the data within the database for easily moving around the database in a single step, then you will need to use blobs. In my experience, the "store it in the database" approach often is the first thought. However, this is rarely the best approach. The exception usually involves security requirements, although even those can often be addressed while storing data via a file system. Cleaning Up Your Mess When you decide to use a file system as storage either for temporary files or the long term, you have to consider clean-up. The problem with using a file system for storage is that it does not clean up or re-use space automatically. You will need to create processes that determine what can be removed/re-used and how that clean up is done. This obstacle is not very different from situations where log files can become large or numerous. However, it is a problem you should keep in mind while architecting your solution.

One of the rarest of decisions we are allowed to make is when it comes to selecting languages and frameworks or maybe even libraries for our project. These decisions are often out of our control. Thus, this is not something with which we may have any experience. While we do have the options to do this for our personal applications, the larger ones usually are almost predetermined concerning the implementation tools. Choosing a Primary Language The choice we least can make is the core language. We are more or less going to be stuck with the corporate language. Failing that, it will be what is popular in the region or what we are most reliable in. Nevertheless, there are situations where we are left to build a case for a primary language for a project. There are several critical factors to consider (stability of the language, whether it is appropriate for the type of solution, availability of programming resources). However, there are other things you should keep in mind while making this decision, as well. These other items include licensing, cost, extensibility, vendor support, user community, and our ability to use the language. Do your homework, and do not merely select that cool language you always wanted to learn. Frameworks and Libraries Out of selecting languages, frameworks, and libraries. The framework seems to be the most common. Yes, we often have a bunch of libraries we use. However, those tend to be roughly the only option. The most common tasks we code are often codified in a library that is used by an overwhelming number of developers with the same need. Think about things like J/N Unit, log4j. Apache commons and similar libraries. These are almost defaults that you will use for your applications. A choice is not really what we can call this. Frameworks can fall into a similar situation as libraries. On the other hand, we often have two or three good options from which to choose. I am thinking about frameworks ranging from database integration to testing to javascript to UI (CSS) and many others. In these cases, we need to look at what problems we most need to solve and whether the frameworks help us get that work done better, faster, or more reliably.

In no particular order, we will look at architecting the database as we continue our tour of software architecture from scratch. There are several key considerations to keep in mind while doing this work. A wrong decision or improper architectural choice can have a far-reaching impact as we build from the backend. Here are some areas that we want to think through as we design our backend solution. Primary Keys One of the most commonly utilized values in a database is the primary key. This is the value that uniquely identifies a record within the database in a specific table. You can limit that uniqueness to the database and even table, or you can make it universally unique (GUID). While there are pros and cons for each approach, our specific solution should dictate the approach we take. Generally speaking, integers that are unique within a table are the simplest solution. Thus, this is also the most index-friendly approach. A GUID can allow you to do complex things with moving around your data. However, it has a higher storage cost and is not as quick to find in an index as an integer in most systems. There are situations where a GUID is required. Therefore, ensure you have defendable reasons for whichever choice you make. Data Types A common mistake by those new to databases is to have large and simple defaults for data types. For example, every string is hundreds of characters in length, and every number is as big and precise as possible. This method is a significant waste of space and resources. While space is cheap and this can work for small databases (like a simple customer contact database or small catalog), this does not scale. Think about simple space calculations. If you allow for 100 characters in length everywhere but often need less than 20 and never more than 80, you are using a lot of empty space. This can be a small 20 bytes per row, but it adds up. A thousand rows will be only 20K of wasted space. However, think about that as you move into millions or billions of rows. Even worse, this calculation was for only one column. The waste adds up very quickly. Space is an issue, but so is interoperability and usability. Dates, in particular, have a broad range of formats, so find one to use as your default and stick to it. These issues are essential in the back-end, but they will re-appear when we look at the front end as well. Naming and Auditing Standards are always helpful for maintenance. The easier it is for a new developer to jump in and feel familiar with the system, the better. There are sweeping architectural decisions to settle on early in the process. These include things like how data auditing will be implemented, naming conventions, and what work will be held in the back-end as opposed to other layers. Architecting the database may seem like a simple task of identifying tables and columns. However, there is much more to be done before you can finish this architectural task.

Every journey starts somewhere. When we architect a solution, we need to decide whether to start at the frontend or the backend. There are pros and cons to each approach. Therefore, we need to look at each project to help us determine which method is best. Let's look at the strengths of each of these starting points. The Frontend Approach The frontend often feels like a comfortable way to start. You can focus on user experience and the visual pieces of the solution. This can hide complexity. On the other hand, it is well suited to things like stories and clickable demos. We will have visual designs and decisions that can easily be shared and assessed. We also are pushing technical issues like validations, manipulations, and similar data issues off until later. Nevertheless, a front-end approach does help us quickly design pieces of the solution that can be discussed and challenges (like validations, manipulations, and rules) highlighted as we see the vision move to something more real. The frontend is not all pretty controls and user-experience widgets. We also will be building out critical data items and interactions as well as the first steps for business rules and logic. Do not underestimate how much of your architecture can be launched from the frontend. Learn more from this article: User Experience Design For Developers. The Backend Approach We all tend to associate the backend with data. Of course, that makes sense as the bulk of the backend is data and storing the same. This approach may feel like a cold implementation that ignores the user experience. However, it is not only a function over form approach. There are many rules and performance decisions that drive this part of the architecture. We will even be starting into user experience decisions like drop-downs or other selection approaches. More importantly, we will be modeling data in a way that will smoothly lead to frontend requirements. A good example is an address field. We look at the ways an address can be stored, and this will directly inform the best approach (or approaches) for the frontend when handling this data. This approach will even help us avoid re-architecting pieces that come from improper assumptions. Of course, proper questions and attention to detail can provide the same result. A Hybrid Solution While we do not touch on it in this episode, you can start from both ends. Your UI decisions can be used to help drive backend architecture and constraints. Likewise, backend decisions can nail down frontend controls and needs. While either approach works on its own, you can always alternate between the two methods to effectively "walk-in" the precision of your architectural design.

The decision of Agile vs Waterfall in software development is not often linked to architecture. However, there is a difference in your approach that depends on the SDLC you choose. This choice is one you will probably be asked to make when starting from scratch. While not entirely the domain of the architect, this decision often seems to be driven by them. Agile vs Waterfall - Different Mindsets The most significant impact this decision will have is the one on your mindset. A waterfall approach front-loads a lot of the architecture decisions. You can define a relatively static and complete architecture without as much fear of substantial changes. This includes common concerns like performance, cost, and extensibility. An agile approach is going to change as you go. That is the nature of this approach to software development. While you can always go back and re-architect the solution, that is still often a costly way to create your application. Instead, your architecture needs to be more flexible and extensible than one in a waterfall project. While you can prioritize performance and a tight architecture with waterfall, Agile demands a priority on flexibility. Architecting Into a Corner One of the most common questions I find myself asking when architecting via Agile is whether something will need to be adaptable or an approach can set in stone. This is a trade-off we often see in building software. We can remove system flexibility to improve performance (even to the point of hard-coding values). However, that can bite us down the road if we suddenly have to support a different requirement. The most common example of this is the desire to store one or more addresses per (client, user, customer, etc.) object. We can model the address as a part of the main object, or we can have a one-to-many relationship. If we choose the one-to-one initially, then we can be forced to recode in the front, middle, and back tiers later in the project, if one-to-many is required. These costly changes are essential to avoid in an Agile environment. A Firm Foundation Is Always Critical While there are different concerns for architecting a solution based on our SDLC choice, there are some excellent first steps in every case. We will almost always need to maintain, modify, and extend our design. This requires us to provide a solid foundation for the architecture that avoids too many fragile structures. The Agile approach may be more unforgiving of bad architectural decisions early on; it seems we will pay the price for those mistakes sooner or later in any case.

We kick off the season on software architecture from scratch with an overview of what we will cover. We also set the table for the assumptions we will make as we walk through our topics. Each topic will cover an area or two of architecting software that we need to address when we start at the beginning. These are not always rocket science. However, details matter, and we need to keep them in mind while mapping out our solution. Software Architecture Requires Questioning We have many ways we can approach creating a solution. However, this season will assume we asked a lot of questions and got enough answers to build out a detailed list of requirements. This is a critical step when creating software and worth its own season of examination. Nevertheless, we will still be asking questions as part of architecting the solution. There are layers of understanding we need to achieve in order to design the best system for any set of requirements. When you think about requirements as answering the "what" questions, then software architecture will need to address the "how." Constraints and Environments We will see themes of questions as we move through this season. One is the idea of constraints. These may be due to resources, user experience needs, and technical considerations. However, we will often go back to questions the help us determine when, where, and how many limitations need to be taken into account with our architecture. The Tip of The Iceberg There is a lot to cover when we tackle architecting a solution. It requires a broad amount of technical knowledge and robust experience. In this episode, we touch on some of the topics we will cover. However, this is just the tip of the iceberg. We will have numerous issues that feed into our architectural decisions and crafting before we get to the end of the season.

As we near the end of a decade, it is time to consider planning for the year ahead. We need to consider the goals we want to achieve and how much better to be a year from now. Here are some ideas to help you plot out your course. Improve Your Workspace Whether you work from home or an office, there are many ways you can improve your daily experience. This can include hardware, office equipment (or supplies), or even software and services. All of these items are likely to improve your productivity. Therefore, there is at least one good reason to pursue these improvements. The more significant ticket items like a new desktop/laptop may seem daunting to pursue. However, think of how much time you can save when your daily work includes less time waiting on your machine. Comfort is another reason for making changes. You will find yourself better able to handle long hours, reduce stress, and maybe even improve your physical well-being. These can all be used to argue for an improvement form your boss. On the other hand, you should find it easy to convince yourself of the ROI on these types of improvements. If you need more convincing, then take a little time to check out this episode. All of these add up to something you should consider in your planning for the year ahead. Learn a Language or Technology There is no end to our need for learning new languages and environments. Sure, we can sit back and try to coast with what we know. However, technology marches on and will pass us by if we fail to stay current. Go ahead and push yourself to learn something new that will make your resume a bit more attractive. An excellent way to improve your skills is to learn them while knocking out a project. You get to complete two goals with one project. This approach can lead to significant gains during the year and maybe even some well-earned time off next December. We want to improve continuously. However, we can afford to take breaks when we have been uber-productive. Include Something Big in Planning for the Year Ahead Our recent episode on big achievements we all can complete is a good starting point for your annual goals. I have taken this on in the past. Yes, these sorts of targets can be scary and intimidating. On the other hand, there is a lot of value and self-satisfaction to be gained by achieving any one of them. Each will also be a valuable addition to your portfolio and resume. The year ahead can easily be your best year yet. Do not ignore these options in your planning for the year ahead. Go ahead and put some impressive goals on your list and see how awesome you can be!

We once again look at finishing strong. Our steady efforts and incremental progress should result in year-end success. However, there always seems to be a need for that final push. No matter how well we plan, the final effort to get across the finish line is a staple of completing a project. This episode is just a little help in cheering you to the end. Pace Out The Final Mile We may have missed the mark on our estimation. However, when we get near the end, it should be easier to estimate what we have left to do. We also know the days or hours we have left before the end of the year. That becomes simple math as we can divide the hours we need to work into those remaining. We can even start today on that work and maybe reduce what we need to do each day by pushing now. That may even get us across the finish line early. When that happens, jump for joy and enjoy the well-earned time off. Lessons Learned From Year-End Success Whether we are successful or not, we will have learned about ourselves and our ability to achieve goals. We may have more proof that steady work will pay off. However, we may have missed our goal and know more about our limits. For example, we may see the last two months of the year are too busy to maintain the same pace we had in February. When we plan ahead we can take those seasonal adjustments to our drive into consideration. There is no need to push ourselves too hard. We still are human and subject to burnout. Enjoy Success One of the greatest joys is a year-end success when it comes early. We all get that endorphin boost when we complete a goal. However, getting it done early is even better. At least in my personal experience. The rare years where I have my goals met by mid-December or earlier are the best. If you are fortunate enough to achieve this, then do not fill in the "free time" with more goals. Revel in your low-stress time and use the positive feeling as an incentive to repeat this in the future.

As we start into our last week of the year, it is a good time for looking back. Therefore, we think it is worthwhile to review what we proposed would be the big technical topics of 2019. The summary of that article is that we suggested five hot topics. Azure/Office 365 Amazon AWS Single-Sign-On/Security Virtual Machines/Containers Blockchain Solutions Our Hits in Looking Back The good news is that each of our five recommendations worked out. Thus, if you learned more and gained experience in these areas during 2019, then you did well. All five of the items we listed have continued to grow and mature. Better yet, there are plenty of jobs related to these technologies so you are more marketable. The Technologies That We Missed In scanning the landscape of articles about 2019, we failed to mention several technologies that were hot. Many of these have moved faster than we thought. However, the speed that these technologies were adopted seems to have surprised many in the industry. Here are the misses we discussed. Internet Of Things Artificial Intelligence 5G Networks Serverless Biometrics Augmented Reality Drones Our Mea Culpa There are some important things to note about these technologies. For example, IoT has moved along at a good pace in the past few years. However, we saw a big market appear for these devices in the areas of home security and automation (Nest, Ring, etc.). The prices have been reduced by Amazon and Google to flood the market. That push has been successful for them. Biometrics, Drones, and Augmented Reality are technologies that have been floated to some extent in the past. Thus, we were not sure this would be the time they would become mainstream. Alas, we were wrong. Nevertheless, the cost of these devices has come down to a point where we see them everywhere. Biometrics have appeared on laptops, phones, and even watches. Drones are everywhere it seems as video quality has improved. Finally, Augmented Reality is getting to a point where the computations required can be done in a useful amount of time. That is our looking back discussion. Therefore, we can start to close the book on 2019 and look to the year and decade ahead. Read More https://www.techrepublic.com/article/top-10-emerging-technologies-of-2019/

We wrap up Thanksgiving week with some recommendations for making the most of that time off. While you may be working frantically on Black Friday, there will be a time where you can take a day. Those days off and vacation times are essential to take advantage of. You can even become a better developer by making the most of your "downtime." Get Some Rest Our lives are incredibly hectic, and the days of often long. That can chip away at our energy levels and even our health. One of the primary goals of a day off should be to get some rest. Take a nap, sleep in, or at least do something mindless. This is a time to get away from your usual grind and recharge your batteries. While there are thought exercises we can do best during these times; there still should be some rest planned. The rest you take can help your morale, attitude, and even your health. Ways that sleep improves productivity Meaningful Length To Vacation Three-day weekends are nice. However, it takes more than an occasional day off to truly rest and regenerate mentally. Most of us need a couple of days to wind down and get used to less stress or deadlines. Thus, spreading our vacation days too thin can defeat the purpose. It is better than "selling" our vacation back to our employer but only a little bit. Make sure you put together a vacation of a week or more at least once or twice a year. That will give you something to look forward to as well as enough time to let your body and mind recover from our typical torrid pace. True Design and Problem Solving I hate to throw in a work-related focus during our time off. However, the periods where we can think without stress and deadlines are the best for processing problems. There is a reason so many well-known problem solvers in history took time away from the world to rest and re-focus. We can do the same while toning down our normal struggles and urgencies. Therefore, the next time you have some time off, take advantage of it. You earned it.