Unbalance Your QlikView Cluster

In this webinar we will discuss:

• Benefits of an unbalanced cluster
• Exploring Distribution Groups
• How to Deploy
• Avoiding pitfalls

Additional comments
Hey everyone, I have a few corrections to my presentation I am going to lay out here for you guys.

1. Things did successfully complete, I think the reason they were misbehaving during the presentation was underpowering of the VMs, and that is my fault, I should have requested more cores for the VMs to be sure I had plenty of CPUs to handle the load with everything, so everything did work exactly as expected once everything properly updated, so the functionality is working perfectly now.  One side note, if you switch between modes, 'regular' and 'unbalanced', you will need to be sure the workorder updates between the QMS and QDS as well, as until that occurs, the QDS is going to run the last selected option it had registered.  An easy way to 'force' things is go to the QMC\System\Setup\QDS Resource\Advanced tab settings and click the Send Workorder button to have the QMS update the QDS nodes with the change to the workorder/task information.  UPDATE: Root cause identified as a clock issue on one of the QDS nodes causing the inconsistency in the QMC and Task Result reads, shutting down the QDS on the node with the clock problem resulted in everything working much better.  Have not determined root cause of the clock problem as yet, but I will post again when I am able to sort that out.  Thought it was Region settings in the OS, but that did not do the trick, so something else is amiss on the one node.  
2. I created some confusion as well regarding the <GraceTimeMinutes>0</GraceTimeMinutes> setting in the DistributionGroupDefinition.Template.xml located in C:\ProgramData\QlikTech\ManagementService\DistributionGroups, where I was mentioning this was how you change the graceful shutdown wait time on the QDS nodes, this is NOT correct, that setting is located in the QVDistributionService.exe.config as <add key="ServiceStopGracetimeInSeconds" value="1800"></add> and will be per QDS node, apologies for confusing you on that one. 
3. When I went to the LoadBalancer.xml file and I noted my confusion as to why things were not updating, I was missing the fact that we only keep the current set of entries, all we do is update the values in those existing rows, so you will not see any historical rows in that file, just the most recent set of values that were assigned, sorry I forgot this is how things updated in this file.  I put a Q&A below on this as well, just FYI.

Q&A - Unbalance Your QlikView Cluster

Q: Can I have 2 different Web services running on top of 2 QVS?
A: Correct, and sorry I did not see this one to show you directly, but you can have as many QVWS resources as you want, you just need a Network Load Balancer to load balance those if you want to have failover/high availability, or if you have separate URLs for different AccessPoints, then it would just be a matter of the users knowing which web server URL they need to hit to see the apps they want to see.  The other thing that controls which apps appear on the web server is going to be in QMC\System\Setup\QVWS resource\AccessPoint tab\Server Connections menu and then the Server dropdown, this is where you can choose which server(s) you want to send applications to that web server.  Use the green + button in that page view to add additional QVS resources to a single web server.
 
Q: Is it possible to have a single QlikView web address/URL determine where the app resides, and the user is directed to the web server?
A: The solution to this would be like above I believe, the only solution I can see to this is you have single or multiple web servers, but all the web servers have all QlikView Server instances attached to them via QMC\System\Setup\QVWS resource\AccessPoint tab\Server Connections menu and then the Server dropdown, and to add the additional nodes, use the green + button over on the right side to add in the additional QlikView Servers. 
The only other way I see to do this would be a custom front-end/portal that would list all the apps and when the user clicks on the one they want they are directed to the correct AccessPoint.aspx URL call.  You can get the URLs from the AccessPoint page by clicking the View Details link on any given application and then right click the client you want to use and choose the Copy Link Address option and paste that into the portal page.
 
Q: Can you talk more about weights in the web server settings? Are they relative to the sum of all other weights? So, if loadeddoc = 5, cpu = 2, and RAM = 3, then is loaded doc 50% of the balancing preferences?
A: This is an excellent question to which I do not have definitive answer off the top of my head, but what I will do is get this one over to the engineers and get the answer, and then either do an Article on it, or get the Help page update for you.  I would expect you are correct in your assumption, but I am not quite sure how to go about testing to be sure.  I would say if you can, test things out, and if you see it not behaving as you would expect, go ahead and open a support case too, as this will allow us to have that to submit a bug to get the engineers to recheck things and then either update the doc or change the functionality.  Apologies for not knowing the details on this one, but I will follow-up on things and do the Article etc.
R&D answer:  The numbers are “normalized”, meaning they are reflecting the relative number amongst themselves. Your example with “loadeddoc = 5, cpu = 2, and RAM = 3” will then give LoadDoc 50% “power”. You are free to set these numbers as you want, “loadeddoc = 2005, cpu = 2, and RAM = 3” will also work but giving so much strength to the LoadDoc that it’s debatable if it’s really Load Balancing.
 
Q: Does Qlik Sense work the same way?  Or is this just applicable to Qlik View?
A: Qlik Sense has this functionality built-in, as you can add as many engine nodes as you wish, and they do not need to be the same size servers.  The issue in QlikView was how we initially did the configuration files etc., everything was expected to be the same, we were able to rewrite that portion of code when doing Sense. 
 
Q: Is there a good way to test QlikView server load balancing, short of pulling the plug on one server in the cluster to see that documents still open by way of the other server(s) in the cluster?
A: Yes, you can always use the QMC\Status\QVSStatistcs view and select the server(s) to see which documents are open on which nodes that way.
 
Q: What's a distribution group?
A: Not exactly sure of the context here, I am assuming the question is based upon the Help for the Distribution Service Clustering, which calls things Publisher Groups, and it is really nothing more than allowing you to put specific tasks into specific 'groups' and then set which QDS nodes those groups are allowed to run upon. 
 
Q: We have many teams who manage their own tasks in the QMC, can I assign one group to Group A and another to Group B?
A: The only way to get this functionality would be to uncluster your QDS resource into individual QDS Resources and set each QDS Resource to a different set of Source Document Folders instead.  The Publisher licensing allows you to run either multiple QDS Resources or 'Clustered' resource, or a combination, you just have to have enough nodes allowed in your Publisher license. 
 
Q: Our document log is too big, so it does not load?
A: I believe you are asking about the new Document Log tab in the QMC in 12.20/November 2017 track, and yes, I do believe there is a limit on the number of rows that are returned by default to avoid hanging the QMC, but this should be adjustable, so you may be able to get more lines, but the question is whether that may negatively impact the QMC performance...  Took me a bit, but I think I found the correct setting: MaxLogRecords, the default value is 1000, and the setting should be visible in the QMS exe.config, which by default is located here: C:\Program Files\QlikView\Management Service\QVManagementService.exe.config
 
Just be careful as to how large you set things, as you may need to change the MaxReceivedMessageSize setting as well to a larger value as well.
 
Q: If you do not specify a group for some tasks, where will it run them?
A: If no Group is Specified, or the 'Any' option is set, that means any QDS node in the cluster will be able to run that task. 
 
Q: If we distribute with Active Directory with a large set of users, then it kind of sets security to the files which is a big list, we could avoid this by creating groups, is there any way the validation is done on the web and access this qvw files thru that which we do?
A: Sorry if I did not get this covered well at the end of the presentation, but you can use groups, which it seems you have noted, but it looks like you may be having an issue in the web server side, which if so, and you are not using NTLM option for the Web Server resource Authentication settings, and you are using Configurable LDAP or one of the other DSPs, this may be the issue, as you have to be sure the Configurable LDAP or the Configurable ODBC etc. are configured for groups in order to be able to do the group resolution.  I hope this is to what you were referring, my apologies if not.
 
Q: Will unbalanced clusters support high availability for automatic failover?
A: When I answered this, I think I only covered the QDS side, but QVS would still handle things as well, the only issue is whether you have enough excess resource capacity in the remaining nodes to handle the additional load from the server that goes down, that is going to be the tricky part in this scenario.  The same would apply for the QDS as well, but we are a little better off there in that we have the engine limitations, so what is likely going to happen is that the reload/distribution window just expands while that node is down unless you have enough excess capacity to be able to pick things up fully.
 
Q: Do we lose task config when we apply new license for clustering publisher?
A: Applying a new Publisher license with additional XS allowed should not cause you to lose anything, the one thing that will cause the tasks to break is if you change the Source Document Folder paths in the QMC for the QDS after you have already configured tasks.  I have an Article I edited regarding this that contains a note on editing the SourceDocumentFolderResource.xml file in the QVPR folder directly to change the path there, as this keeps the GUIDs intact in the QVPR files and will ensure nothing breaks.  The article number and name are: 14618 and Migration/Upgrade from Qlikview 11.20 to Qlikview 12.  My note is step 10 in the Migration to New Machine section.
 
Q: What could be the cause of getting "Index out of range" error after upgrade to QV 12.20 SR 4?
A: I am not sure I can answer this without further context, but article 46385 Index out of range error when trying to open Qlikview task, may be what you need here.  You can try the manual workorder push, but now that I reviewed this one, in most of the cases I have encountered, we have had to rebuild the QDS App Data folder and/or the QMS Workorders folder to get everything back up and running properly.  If you cannot get it sorted, please open a support case with us, so we can have a look, but the Article should give you what you need.
 
Q: How unbalanced QDS will work with EDX, should there be any config other than selecting Distribution group?
A: An EDX triggered task is still the same base task information as any other task, the only difference is the trigger, so you are correct, where it runs will depend upon which Group you attach it to etc.
 
Q: Is number of nodes limited by license file?
A: Yes, for QlikView Server, the number of nodes is represented by the following setting in the QVS LEF.txt file: NUMBER_OF_CLUSTER_NODES and the file is located in C:\ProgramData\QlikTech.
The number of nodes for the Distribution Service/Publisher will be the NUMBER_OF_XS setting in the Publisher LEF.txt located in C:\ProgramData\QlikTech\ManagementService by default.
 
Q: If I have qvw files +4gb or heavy applications, can I set unbalanced node optimized for RAM/memory? the parameters should be 10?
A: I assume you mean the QVS cluster setup in this case, and yes, you can change those algorithm settings to any value you want, a bit of trial and error may be necessary to get things tuned the exact way you want them.  Another thing of which I am unsure here is what the max values are of those, I think I would stick to 0-9 on them, and I will make a note to inquire about that and do a new Article on those setting values.
 
Q: How to determine correct ID's we have seen in the example config file?
A: the GUIDs are populated by the backend automatically when the environment comes online, and if you make any changes, those should be reflected in that Template file after the changes are committed in the QMC.  If they are not visible right away, try restarting the QMS service as that should definitely get everything to sync back up, and you should see those changes now reflected.  We did see that when you rename the Template file, the service does recreate a new one as well, so you could also just delete the old one, and a new file should create right away, or when you restart the QMS service.
 
Q: Do you have any documentation for clustering standard way 4 servers QV environment?
A: The best place to always start is the Help, this is link to 12.20/November 2017 release, but things are pretty much the same since 11.20 that I can recall: https://help.qlik.com/en-US/qlikview/November2017/Subsystems/Server/Content/QlikView-Server/QVSRM_Clustering_Servers.htm  If you are missing anything, try searching the Support Knowledgebase as well, as there may be something more specific there as well.
 
Q: Can groups be configured by time of day?  e.g. between 6am-9am only Group A can run on Publisher A, but all other times any job can run on Publisher A
A: I do not believe things are that granular at this point, if you want us to put in an Ideation on that, we can do so, just open a Support case and let us know you would like to see that functionality, and we can submit it, but given I really do not know of anyone that has used this feature yet, my hunch would be the votes would likely not be there on this one to get it to a point where product management would consider it. 
 
Q: Can we add priority to a Task?
A: The closest thing would be the Dedicated setting, but I have to figure that out, as I thought that was going to be in the task settings somewhere, but I did not see it, so I am not sure how to configure that myself at the moment.  I will work on getting the things that did not work figured out and probably do an Article as well as consider opening a doc bug as well if need be to improve things there too. 
 
Q: Why does the QDS LoadBalancer.xml file not show history?
A: I figured out the entire file is updating, we are not appending new rows, so if you watch in NotePad++, if you wait a few seconds and click back in the open file, NotePad++ will ask you if you want to see the updated info, and what you see are the same lines, one for each node in the cluster, and the rows information is what is updated, sorry I forgot this during the Q&A, but things were working, I just forgot the rows are updated complete every 4 seconds or so.