Component View of Webapp design

 

webapp componets

 

I was thinking about how web applications are designed and what are the different components and languages involved.

Java Script is most popularly used these days for small and mid size web applications and python for enterprise applications.

Angular.js is the single page web app design framework which runs the code inside browser.

Node.js is the backend framework which does database queries and simple data fetching tasks. The code is written in java script.

Python web application framework like flask, django are supported by uWSGI standards, and these are plugged in as modules for uWSGI servers.  All time-consuming tasks are queued using celery task manager framework and these are processed by a set of backend worker threads.

Nginx is used as a routing server for all the different traffic from users on a single IP and port combination. It routes the request to either node.js server or uWSGI server based on the request types.

 

Advertisements

Component Testing

  • Functional testing for APIs
    • optional parameter testing
    • Parameter testing: Size boundaries, accepted and not accepted values, max and min values, empty values/strings/array/datatypes
    •  Combination of APIs testing, with a valid and invalid sequence of operations. (Should not lead to crash)
    • Random data instead of fixed/pre-defined data
    •  Corrupted parameters, break the protocol sequence.
  • Race testing
  • Load and stress testing
  • Harden testing
    • Start multiple instances of the same service on same port
    •  connection tests,  expect failures after exceeding the promised number of connections.
    • Corrupt config files and start the service.  Service fails to start.
    • Longevity tests, Run IO for 24 hours, should not lead to hangs, crashes.  Measure Memory and CPU usage patterns.
    • Shutdown or restart service while IO is in progress
    • Shutdown/restart the backend services  while IO is progress/ not in progress  (expected connection failures)
  • Error Injection tests,  to test the timeouts
    • Inject slowness in the service (to test client timeouts), slowness while establishing a connection, slowness after sending headers
      slowness after sending partial data, slowness in acknowledging responses.
    •  Inject slowness in the client (to test front-end timeouts).

Software Engineer Job interview : Interviewer goals and duties

Recently I opened a requirement for a principal software engineer (with 8-12 years experience), I asked my engineers to take the technical interviews.  After every interview, I ask my engineers a question, “can we hire this candidate?”.  I don’t get a straight answer from these interviewers. They say, he is good in some aspects, just ok in other aspects and candidate could not give a correct answer for that problem…..

Then I asked these questions to them:

  1. What did go well?
  2. What did not go well?
  3. Is he capable of doing the work you do?
  4. How easy to work with this guy?
  5. Score out of 5?

Even with these questions, there are some dilemmas to hire or not hire. Then I understood that these interviewers are going without any goals and they just wanted to get a good feeling of the interview and expecting correct answers to the questions they ask. I came up with some inputs to make their decision process easier.

  • Timebox your interviews for an hour. You can split your time:
    • 2min on ice breaking, no discussion on the salary, no long discussions on the product. If he insists more, tell them that hiring manager will discuss with you.
    • 2-3min: explain the problem
    • 2-3min: make sure he understands the problem.
    • Rest is based on what you are asking and your good guess on, how long it takes.
    • Don’t extend beyond 1 hour.
  • You can ask questions on below categories.
    • The coding interview is to test the ability to write a program for a given problem. It should not be a complex problem. Solution can be identified in say 5-10min and coding can be done in 30-40min. Goal:  efficient solution,  he can write modular code, handles error scenarios, uses the language constructs very well, ability to test his own code.
      • The candidate has to articulate the solution first.  You need to answer all the requirement gathering questions that candidate asks you.
      • Have a guess on how much time it takes and give that much time to write the code.
      • Once he/she writes the code,  see if he/she can come up with some test cases (boundary, negative tests) and apply on his/her code.
      • If he can’t finish, ask him if he wants to spend some more time to write the solution. Let the candidate yield to his inabilities.
      • You need to have few set of coding related questions prepared and have a good guess on how much time it takes to come up with the solution & coding.
      • If you think he/she is not solving the problem, give them a test case and ask if his/her solution addresses the test case.
    • Complex Problem-solving Interview. It takes time to come up with the solutions, it may take 30 mins. Goals: Candidate understands the problem and articulate it, approach to problem: methodical or random, uses his algorithm and data structure skills
      • Idea is that he can split the problem into sub-problems
      • Applies his knowledge of Algorithms and Data structures.
      • Identifies the corner cases and other possible solutions.
      • As an interviewer, your job is to help him answers related to requirements and hints in some cases.
      • Coding is not the criteria.
    • Knowledge interview questions. Goals:  Knows what he claims,  his/her knowledge is useful for us.
      • Example: knows what is a threading model, knows the importance of CPU/ memory utilization, have an understanding of replication and importance of encryption, know distributed databases work,  what is async IO.
    • Design interview questions. Goal:  Candidate has to collect the requirements from you,  he should be able to list the problems that he needs to solve,  can draw a component diagram, Can write simple API,  able to use knowledge. 
      • These are open-ended questions, no fixed answers.  He should collect the requirements from you.
      • Ability to understand system limitations,  uses his knowledge to subdivide the problems.
      • You can focus either on solving subproblems (or) You can focus on High-level design (or) You can focus on API design (or) You can focus on areas where you and the candidate have more knowledge.
      • Your task is to identify such design problem.

 

SAFe Product Owner role

Recently our company adopted SAFe agile framework for our BU.  I was asked to take the Product Owner role. Apart from my People Management job,  I have to take care of this role. Some changes I observed:

Before PO role:
————————

  • I was doing coding
  • I was doing code reviews
  • I was participating in Design discussions
  • I was identifying the list of task to do to deliver a component.
  • I was assigning the tasks to the team.
  • I was taking the ownership of some of the tasks depending on the urgency and delivery times.
  • I was mentoring the team members on technology and career related areas.
  • I was spending time with team to debug the issue. I enjoyed it the most. This was my core competency.
  • I was managing people (I have five people reporting to me)

With PO role:
——————–

  • Understand the PM User stories and discuss with System Architect.
  •  I am identifying the User Stories for the scrum team.
  •  I am writing a clearly defined Acceptance Criteria (This is the timing consuming task, This is the key for product success.). This is the critical job.
  •  I am discussing with team on defined User stories and split them into small user stories.
  •  I am answering team’s question related to the User story. (Mainly the “What” part, Some “How” part as well). I need to know the How part in some user stories, For example, the communication between two components. I should not care about HTTP/GRPC/S3. If team chooses a User story to go with a S3, I need to have domain knowledge to address the AC.
  • I am attending the Ceremonies, Backlog grooming, Sprint Planning, Design discussions, Sprint Reviews.

I see there is need for seperate PM and PO roles. Some of the task sharing between these two roles are:

 

Role Responsibility
Product manager Tracks the overall market and competitive dynamics.
Product manager Manages the long term roadmap, involving sales, marketing, implementations, clients, prospects, partners, and other groups. The ideas are expressed as epics.
Product manager Attends iteration demos and some stand-ups.
Product manager Supports other non-technical organizations (such as sales, marketing, and channel).
Product owner Leads requirements gathering effort on the epics as needed — consulting with product management, implementations, clients, and other stakeholders.
Product owner Documents story details based on the epics and after review with development.
Product owner Attends scrum meetings including standups, retrospectives, and demos.
Product owner Leads backlog grooming to decompose/estimate stories.
Product owner Creates mockups and works with UX on design.
Product owner Answer questions from developers, clarifies requirements, etc.
Product owner Documents the new feature for implementations and release notes.
Both Write acceptance criteria.
Both Demonstrate latest iteration to customers (pre-release) and gathers feedback.

 

 

POC vs Product

POC

  • Focus is on functionality, Algorithms, data structures.  no need to handle all error cases.
  • Proving performance
  • Need reliability to the extent of not having too many issues/bugs while doing performance tests/ basic functionality tests.
  • Test coverage is limited to End to End testing.
  • No need for unit testing for every function/class.
  • No integration with management layer (UI)
  • Have automation to reduce the setup related tasks. (use Ansible playbooks/ shell scripts/ python scripts)
  • Have clear document on what functionality is skipped and what needs to be taken care in the productization phase. ( requirements, features and functionality)

Productization

  • All error handling.
  • Log messages, and log file management
  • Stats to get insights into the product layers.  (need for debugging purpose)
  • Stats to customer
  • Management (Tools or UI) layer integration.
  • Class/Function level unit testing.
  • Sub component level testing.  Make sure there are no race conditions and memory leaks.
  • Integration testing (End to End testing)
  • Performance related tests. Stress and Load testing.
  • All possible functionality related testing at each level. Both Positive and Negative tests. Ask developers to get the infrastructure to induce errors.
  • Tools/Utilities to manage and monitor the product.
  • Call Home feature, where events are sent from customer machine to the  Servicing company.
  • Notifying users on specific events like failures/warnings.
  • Testing on real Hardware.

Code Review Guidelines

Most of the cases I have seen engineers are not doing efficient reviews because it’s not really their code (someone’s code.) or no time to do.  The mindset is very important during code review.

Importance of Code Review:

  • This saves a lot of time in fixing bugs so that we can release the product in time.
  • This is the time engineers learn/teach effective programming practices, language semantics/constructs, new libraries, and algorithms.
  • If an engineer finds bugs during the code review time, it boosts his confidence. This is the clear sign that engineer is matured, technically strong and experienced enough.
  • During the review, a whole lot of implementation details are discussed with the owner of the code,  for other members, it’s a quick way to learn what other team members are doing and what is going into the product.
  • You can learn a new language quickly if you are new to it.

The owners of the code need to get his code reviewed from at least two colleagues.  The code reviewers have to keep below in mind while reviewing. If the changes are huge, a code walk of changes needs to be done by the owner.

  • Functionality is implemented as discussed.
  • Handle Failure cases all over the code.
  • Have enough comments in the code.
  • Check for proper copyright headers in the code.
  • Does the code scales for future requirements.
  • Product does not break during Upgrade and Downgrade scenarios
  • Does it have all the stats covered for debuggability?
  • Logging is sufficient to pinpoint the issue in the code.
  • Adhering to language Coding guidelines
  • The performance will not be degraded.
  • Memory allocations and frees are handled properly.
  • In case, the implementation involves multiple threads,  make sure code is not making CPUs 100% utilized.  Also look for race conditions.
  • If you think the new code is too risky, ask for more unit/component test coverage.
  • Check if the security is compromised as part of the new code.
  • Make sure the developers not copying the code from sensible licensed public code like GPL licensed.
  • See if time outs are incorporated in IO path so that I will not hang.
  • If the developer is using open source libraries ask how stable they are and attributions are listed in release notes.