I am running a distributed test of an application with JMeter. The slaves are two computers from azure with 320GB RAM each. The heap is increased to 284GB in the .bat-files.
When running the test for 100 users, there are no issues. When running the test with 1000 users, all threads are started correctly, several processes are completed (I checked my database), but no threads ever finishes (see image from command prompt). It was in this state for at least 2 hours before I ended the test forcefully.
Screenshot of the commandprompt when running the test for 1000 users
Here is a snippet from the log from jmeter.server.txt file from the slace machines:
java.rmi.ConnectException: Connection refused to host: 10.0.0.4; nested exception is:
java.net.ConnectException: Connection refused: connect
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(Unknown Source) ~[?:1.8.0_401]
at sun.rmi.transport.tcp.TCPChannel.createConnection(Unknown Source) ~[?:1.8.0_401]
at sun.rmi.transport.tcp.TCPChannel.newConnection(Unknown Source) ~[?:1.8.0_401]
at sun.rmi.server.UnicastRef.invoke(Unknown Source) ~[?:1.8.0_401]
at java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(Unknown Source) ~[?:1.8.0_401]
at java.rmi.server.RemoteObjectInvocationHandler.invoke(Unknown Source) ~[?:1.8.0_401]
at com.sun.proxy.$Proxy20.processBatch(Unknown Source) ~[?:?]
at org.apache.jmeter.samplers.BatchSampleSender.sampleOccurred(BatchSampleSender.java:182) [ApacheJMeter_core.jar:5.6.3]
at org.apache.jmeter.samplers.DataStrippingSampleSender.sampleOccurred(DataStrippingSampleSender.java:106) [ApacheJMeter_core.jar:5.6.3]
at org.apache.jmeter.samplers.RemoteListenerWrapper.sampleOccurred(RemoteListenerWrapper.java:94) [ApacheJMeter_core.jar:5.6.3]
at org.apache.jmeter.threads.ListenerNotifier.notifyListeners(ListenerNotifier.java:58) [ApacheJMeter_core.jar:5.6.3]
at org.apache.jmeter.threads.JMeterThread.notifyListeners(JMeterThread.java:1042) [ApacheJMeter_core.jar:5.6.3]
at org.apache.jmeter.threads.JMeterThread.executeSamplePackage(JMeterThread.java:591) [ApacheJMeter_core.jar:5.6.3]
at org.apache.jmeter.threads.JMeterThread.processSampler(JMeterThread.java:501) [ApacheJMeter_core.jar:5.6.3]
at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:268) [ApacheJMeter_core.jar:5.6.3]
at java.lang.Thread.run(Unknown Source) [?:1.8.0_401]
Caused by: java.net.ConnectException: Connection refused: connect
at java.net.DualStackPlainSocketImpl.connect0(Native Method) ~[?:1.8.0_401]
at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source) ~[?:1.8.0_401]
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) ~[?:1.8.0_401]
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) ~[?:1.8.0_401]
at java.net.AbstractPlainSocketImpl.connect(Unknown Source) ~[?:1.8.0_401]
at java.net.PlainSocketImpl.connect(Unknown Source) ~[?:1.8.0_401]
at java.net.SocksSocketImpl.connect(Unknown Source) ~[?:1.8.0_401]
at java.net.Socket.connect(Unknown Source) ~[?:1.8.0_401]
at java.net.Socket.connect(Unknown Source) ~[?:1.8.0_401]
at java.net.Socket.<init>(Unknown Source) ~[?:1.8.0_401]
at java.net.Socket.<init>(Unknown Source) ~[?:1.8.0_401]
at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(Unknown Source) ~[?:1.8.0_401]
at sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(Unknown Source) ~[?:1.8.0_401]
I have increases HEAP size. The script contains no listeners, but when running the script from the master machine, we do generate an HTML report and a csv file.
Server.rmi.ssl.disable is set to true for an easier connection setup between the master and slaves.