diff --git a/AUTHORS b/AUTHORS index 566d88618..28e89ca3e 100644 --- a/AUTHORS +++ b/AUTHORS @@ -46,7 +46,7 @@ Written in 2015 by Sam Hamilton - samhamilton Written in 2015 by Leonardo Carvalho - CarvalhoLeonardo Written in 2015 by Swapnil M Mane - swapnilmmane Written in 2015 by Anton Akhiar - akhiar -Written in 2015-2018 by Jens Hardings - jenshp +Written in 2015-2023 by Jens Hardings - jenshp Written in 2016 by Shifeng Zhang - zhangshifeng Written in 2016 by Scott Gray - lektran Written in 2016 by Mark Haney - mphaney @@ -54,12 +54,14 @@ Written in 2016 by Qiushi Yan - yanqiushi Written in 2017 by Oleg Andrieiev - oandreyev Written in 2018 by Zhang Wei - zhangwei1979 Written in 2018 by Nirendra Singh - nirendra10695 -Written in 2018-2021 by Ayman Abi Abdallah - aabiabdallah +Written in 2018-2023 by Ayman Abi Abdallah - aabiabdallah Written in 2019 by Daniel Taylor - danieltaylor-nz Written in 2020 by Jacob Barnes - Tellan Written in 2020 by Amir Anjomshoaa - amiranjom Written in 2021 by Deepak Dixit - dixitdeepak Written in 2021 by Taher Alkhateeb - pythys +Written in 2022 by Zhang Wei - hellozhangwei +Written in 2023 by Rohit Pawar - rohitpawar2811 =========================================================================== @@ -92,7 +94,7 @@ Written in 2015 by Jimmy Shen - shendepu Written in 2015-2016 by Sam Hamilton - samhamilton Written in 2015 by Leonardo Carvalho - CarvalhoLeonardo Written in 2015 by Anton Akhiar - akhiar -Written in 2015-2016 by Jens Hardings - jenshp +Written in 2015-2023 by Jens Hardings - jenshp Written in 2016 by Shifeng Zhang - zhangshifeng Written in 2016 by Scott Gray - lektran Written in 2016 by Mark Haney - mphaney @@ -100,11 +102,13 @@ Written in 2016 by Qiushi Yan - yanqiushi Written in 2017 by Oleg Andrieiev - oandreyev Written in 2018 by Zhang Wei - zhangwei1979 Written in 2018 by Nirendra Singh - nirendra10695 -Written in 2018-2020 by Ayman Abi Abdallah - aabiabdallah +Written in 2018-2023 by Ayman Abi Abdallah - aabiabdallah Written in 2019 by Daniel Taylor - danieltaylor-nz Written in 2020 by Jacob Barnes - Tellan Written in 2020 by Amir Anjomshoaa - amiranjom Written in 2021 by Deepak Dixit - dixitdeepak Written in 2021 by Taher Alkhateeb - pythys +Written in 2022 by Zhang Wei - hellozhangwei +Written in 2023 by Rohit Pawar - rohitpawar2811 =========================================================================== diff --git a/addons.xml b/addons.xml index e5a926a14..9197f386e 100644 --- a/addons.xml +++ b/addons.xml @@ -47,10 +47,12 @@ + + @@ -77,7 +79,10 @@ + + + @@ -90,6 +95,8 @@ + + - + + + diff --git a/framework/entity/ResourceEntities.xml b/framework/entity/ResourceEntities.xml index 349e5139a..e975c91ef 100644 --- a/framework/entity/ResourceEntities.xml +++ b/framework/entity/ResourceEntities.xml @@ -145,7 +145,7 @@ along with this software (see the LICENSE.md file). If not, see - + @@ -186,6 +186,7 @@ along with this software (see the LICENSE.md file). If not, see + The date/time a blog post within a category was sent by email or other means. diff --git a/framework/entity/ScreenEntities.xml b/framework/entity/ScreenEntities.xml index 7f04eb37f..cc296a3e1 100644 --- a/framework/entity/ScreenEntities.xml +++ b/framework/entity/ScreenEntities.xml @@ -139,6 +139,7 @@ along with this software (see the LICENSE.md file). If not, see + diff --git a/framework/entity/SecurityEntities.xml b/framework/entity/SecurityEntities.xml index 71cc6405b..e5e7f8a26 100644 --- a/framework/entity/SecurityEntities.xml +++ b/framework/entity/SecurityEntities.xml @@ -513,6 +513,7 @@ along with this software (see the LICENSE.md file). If not, see + diff --git a/framework/entity/ServerEntities.xml b/framework/entity/ServerEntities.xml index f6f0481c9..e9b63af25 100644 --- a/framework/entity/ServerEntities.xml +++ b/framework/entity/ServerEntities.xml @@ -116,9 +116,9 @@ along with this software (see the LICENSE.md file). If not, see - + - + @@ -196,6 +196,9 @@ along with this software (see the LICENSE.md file). If not, see + + + diff --git a/framework/entity/ServiceEntities.xml b/framework/entity/ServiceEntities.xml index 81696e90f..7b51a3ab3 100644 --- a/framework/entity/ServiceEntities.xml +++ b/framework/entity/ServiceEntities.xml @@ -383,6 +383,7 @@ along with this software (see the LICENSE.md file). If not, see + diff --git a/framework/service/org/moqui/impl/EntitySyncServices.xml b/framework/service/org/moqui/impl/EntitySyncServices.xml index bb0d8cad1..098bdf39d 100644 --- a/framework/service/org/moqui/impl/EntitySyncServices.xml +++ b/framework/service/org/moqui/impl/EntitySyncServices.xml @@ -236,13 +236,10 @@ along with this software (see the LICENSE.md file). If not, see // ec.logger.warn("=========== get#EntitySyncData entityName=${entryMap.entityName} count=${currentCount} find=${find}") if (currentCount > 0) { - EntityListIterator resultEli = find.iterator() - try { + find.iterator().withCloseable ({resultEli -> int levels = entryMap.dependents ? 2 : 0 resultEli.writeXmlText((Writer) entityWriter, null, levels) - } finally { - resultEli.close() - } + }) } } diff --git a/framework/service/org/moqui/impl/InstanceServices.xml b/framework/service/org/moqui/impl/InstanceServices.xml index 7071d77fa..dacbd18ca 100644 --- a/framework/service/org/moqui/impl/InstanceServices.xml +++ b/framework/service/org/moqui/impl/InstanceServices.xml @@ -214,8 +214,13 @@ along with this software (see the LICENSE.md file). If not, see - - + + + @@ -281,8 +286,9 @@ along with this software (see the LICENSE.md file). If not, see - https://docs.docker.com/engine/security/https/ --> - + + + + + + + + + + + + + + + + + + + + + + + + + + + Authentication code is not valid + + + + + + + + @@ -377,7 +411,7 @@ along with this software (see the LICENSE.md file). If not, see - A reset password was sent by email to ${userAccount.emailAddress}. This password may only be used to change your password. Your current password is still valid. + A reset password was sent to the email of username ${userAccount.username}. This password may only be used to change your password. Your current password is still valid. You must change your password before login. diff --git a/framework/src/main/groovy/org/moqui/impl/context/ArtifactExecutionFacadeImpl.groovy b/framework/src/main/groovy/org/moqui/impl/context/ArtifactExecutionFacadeImpl.groovy index 82e9acb3d..5edf7ab96 100644 --- a/framework/src/main/groovy/org/moqui/impl/context/ArtifactExecutionFacadeImpl.groovy +++ b/framework/src/main/groovy/org/moqui/impl/context/ArtifactExecutionFacadeImpl.groovy @@ -172,6 +172,10 @@ class ArtifactExecutionFacadeImpl implements ArtifactExecutionFacade { return sw.toString() } + ArtifactExecutionInfoImpl.ArtifactTypeStats getArtifactTypeStats() { + return ArtifactExecutionInfoImpl.getArtifactTypeStats(artifactExecutionInfoHistory) + } + void logProfilingDetail() { if (!logger.isInfoEnabled()) return diff --git a/framework/src/main/groovy/org/moqui/impl/context/ArtifactExecutionInfoImpl.java b/framework/src/main/groovy/org/moqui/impl/context/ArtifactExecutionInfoImpl.java index 5880cf390..67fe40d54 100644 --- a/framework/src/main/groovy/org/moqui/impl/context/ArtifactExecutionInfoImpl.java +++ b/framework/src/main/groovy/org/moqui/impl/context/ArtifactExecutionInfoImpl.java @@ -17,6 +17,8 @@ import org.moqui.impl.entity.EntityValueBase; import org.moqui.util.CollectionUtilities; import org.moqui.util.StringUtilities; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import java.io.IOException; import java.io.StringWriter; @@ -26,6 +28,7 @@ import java.util.*; public class ArtifactExecutionInfoImpl implements ArtifactExecutionInfo { + protected final static Logger logger = LoggerFactory.getLogger(ArtifactExecutionInfoImpl.class); // NOTE: these need to be in a Map instead of the DB because Enumeration records may not yet be loaded private final static Map artifactTypeDescriptionMap = new EnumMap<>(ArtifactType.class); @@ -153,6 +156,7 @@ public void copyAuthorizedInfo(ArtifactExecutionInfoImpl aeii) { @Override public long getRunningTime() { return endTimeNanos != 0 ? endTimeNanos - startTimeNanos : 0; } public double getRunningTimeMillisDouble() { return (endTimeNanos != 0 ? endTimeNanos - startTimeNanos : 0) / 1000000.0; } + public long getRunningTimeMillisLong() { return Math.round((endTimeNanos != 0 ? endTimeNanos - startTimeNanos : 0) / 1000000.0); } private void calcChildTime(boolean recurse) { childrenRunningTime = 0; if (childList != null) for (ArtifactExecutionInfoImpl aeii: childList) { @@ -211,6 +215,128 @@ public void print(Writer writer, int level, boolean children) { } private String getKeyString() { return nameInternal + ":" + internalTypeEnum.name() + ":" + internalActionEnum.name() + ":" + actionDetail; } + private String getKeyStringNoName() { return internalTypeEnum.name() + ":" + internalActionEnum.name() + ":" + actionDetail; } + + public static class ArtifactTypeStats { + + public int screenCount = 0, screenTransCount = 0, screenContentCount = 0, restPathCount = 0, + serviceViewCount = 0, serviceOtherCount = 0, + entityFindOneCount = 0, entityFindListCount = 0, entityFindIteratorCount = 0, entityFindCountCount = 0, + entityCreateCount = 0, entityUpdateCount = 0, entityDeleteCount = 0; + public long screenTime = 0, screenTransTime = 0, screenContentTime = 0, restPathTime = 0, + serviceViewTime = 0, serviceOtherTime = 0, + entityFindOneTime = 0, entityFindListTime = 0, entityFindIteratorTime = 0, entityFindCountTime = 0, + entityCreateTime = 0, entityUpdateTime = 0, entityDeleteTime = 0; + public void add(ArtifactTypeStats that) { + if (that == null) return; + + screenCount += that.screenCount; screenTransCount += that.screenTransCount; + screenContentCount += that.screenContentCount; restPathCount += that.restPathCount; + serviceViewCount += that.serviceViewCount; serviceOtherCount += that.serviceOtherCount; + entityFindOneCount += that.entityFindOneCount; entityFindListCount += that.entityFindListCount; + entityFindIteratorCount += that.entityFindIteratorCount; entityFindCountCount += that.entityFindCountCount; + entityCreateCount += that.entityCreateCount; entityUpdateCount += that.entityUpdateCount; + entityDeleteCount += that.entityDeleteCount; + + screenTime += that.screenTime; screenTransTime += that.screenTransTime; + screenContentTime += that.screenContentTime; restPathTime += that.restPathTime; + serviceViewTime += that.serviceViewTime; serviceOtherTime += that.serviceOtherTime; + entityFindOneTime += that.entityFindOneTime; entityFindListTime += that.entityFindListTime; + entityFindIteratorTime += that.entityFindIteratorTime; entityFindCountTime += that.entityFindCountTime; + entityCreateTime += that.entityCreateTime; entityUpdateTime += that.entityUpdateTime; + entityDeleteTime += that.entityDeleteTime; + } + public ArtifactTypeStats cloneStats(ArtifactTypeStats that) { + ArtifactTypeStats newStats = new ArtifactTypeStats(); + newStats.add(that); + return newStats; + } + } + static ArtifactTypeStats getArtifactTypeStats(ArrayList aeiiList) { + ArtifactTypeStats stats = new ArtifactTypeStats(); + addArtifactTypeStats(aeiiList, stats); + return stats; + } + static void addArtifactTypeStats(ArrayList aeiiList, ArtifactTypeStats stats) { + if (aeiiList == null) return; + int aeiiListSize = aeiiList.size(); + for (int i = 0; i < aeiiListSize; i++) { + ArtifactExecutionInfoImpl aeii = aeiiList.get(i); + // tight loop, use switch instead of if on these enums for much better performance; run fast for use in on the fly accumulators + switch (aeii.internalTypeEnum) { + case AT_ENTITY: + switch (aeii.internalActionEnum) { + case AUTHZA_VIEW: + if (aeii.actionDetail != null && !aeii.actionDetail.isEmpty()) { + char first = aeii.actionDetail.charAt(0); + switch (first) { + case 'o': // one + case 'r': // refresh + stats.entityFindOneCount++; + stats.entityFindOneTime += aeii.getRunningTime(); + break; + case 'l': // list + stats.entityFindListCount++; + stats.entityFindListTime += aeii.getRunningTime(); + break; + case 'i': // iterator + stats.entityFindIteratorCount++; + stats.entityFindIteratorTime += aeii.getRunningTime(); + break; + case 'c': // count + stats.entityFindCountCount++; + stats.entityFindCountTime += aeii.getRunningTime(); + break; + } + } else { + logger.warn("entity view with no detail " + aeii.toBasicString()); + } + break; + case AUTHZA_CREATE: + stats.entityCreateCount++; + stats.entityCreateTime += aeii.getRunningTime(); + break; + case AUTHZA_UPDATE: + stats.entityUpdateCount++; + stats.entityUpdateTime += aeii.getRunningTime(); + break; + case AUTHZA_DELETE: + stats.entityDeleteCount++; + stats.entityDeleteTime += aeii.getRunningTime(); + break; + } + break; + case AT_SERVICE: + if (aeii.internalActionEnum == AUTHZA_VIEW) { + stats.serviceViewCount++; + stats.serviceViewTime += aeii.getRunningTime(); + } else { + stats.serviceOtherCount++; + stats.serviceOtherTime += aeii.getRunningTime(); + } + break; + case AT_XML_SCREEN: + stats.screenCount++; + stats.screenTime += aeii.getRunningTime(); + break; + case AT_XML_SCREEN_TRANS: + stats.screenTransCount++; + stats.screenTransTime += aeii.getRunningTime(); + break; + case AT_XML_SCREEN_CONTENT: + stats.screenContentCount++; + stats.screenContentTime += aeii.getRunningTime(); + break; + case AT_REST_PATH: + stats.restPathCount++; + stats.restPathTime += aeii.getRunningTime(); + break; + } + + // this aeii is done, how about children? + addArtifactTypeStats(aeii.childList, stats); + } + } @SuppressWarnings("unchecked") static List> hotSpotByTime(List aeiiList, boolean ownTime, String orderBy) { diff --git a/framework/src/main/groovy/org/moqui/impl/context/ContextJavaUtil.java b/framework/src/main/groovy/org/moqui/impl/context/ContextJavaUtil.java index ea72322e4..ee4a33b44 100644 --- a/framework/src/main/groovy/org/moqui/impl/context/ContextJavaUtil.java +++ b/framework/src/main/groovy/org/moqui/impl/context/ContextJavaUtil.java @@ -695,17 +695,17 @@ static class ScheduledThreadFactory implements ThreadFactory { private final AtomicInteger threadNumber = new AtomicInteger(1); public Thread newThread(Runnable r) { return new Thread(workerGroup, r, "MoquiScheduled-" + threadNumber.getAndIncrement()); } } - static class CustomScheduledTask implements RunnableScheduledFuture { + public static class CustomScheduledTask implements RunnableScheduledFuture { public final Runnable runnable; public final Callable callable; public final RunnableScheduledFuture future; - CustomScheduledTask(Runnable runnable, RunnableScheduledFuture future) { + public CustomScheduledTask(Runnable runnable, RunnableScheduledFuture future) { this.runnable = runnable; this.callable = null; this.future = future; } - CustomScheduledTask(Callable callable, RunnableScheduledFuture future) { + public CustomScheduledTask(Callable callable, RunnableScheduledFuture future) { this.runnable = null; this.callable = callable; this.future = future; @@ -729,16 +729,20 @@ static class CustomScheduledTask implements RunnableScheduledFuture { @Override public V get() throws InterruptedException, ExecutionException { return future.get(); } @Override public V get(long l, @NotNull TimeUnit timeUnit) throws InterruptedException, ExecutionException, TimeoutException { - return get(l, timeUnit); } + return future.get(l, timeUnit); + } @Override public String toString() { return "CustomScheduledTask " + (runnable != null ? runnable.getClass().getName() : (callable != null ? callable.getClass().getName() : "[no Runnable or Callable!]")); } } - static class CustomScheduledExecutor extends ScheduledThreadPoolExecutor { + public static class CustomScheduledExecutor extends ScheduledThreadPoolExecutor { public CustomScheduledExecutor(int coreThreads) { super(coreThreads, new ScheduledThreadFactory()); } + public CustomScheduledExecutor(int coreThreads, ThreadFactory threadFactory) { + super(coreThreads, threadFactory); + } protected RunnableScheduledFuture decorateTask(Runnable r, RunnableScheduledFuture task) { return new CustomScheduledTask(r, task); } diff --git a/framework/src/main/groovy/org/moqui/impl/context/ElasticFacadeImpl.groovy b/framework/src/main/groovy/org/moqui/impl/context/ElasticFacadeImpl.groovy index 2d30bdd1d..9b2b77107 100644 --- a/framework/src/main/groovy/org/moqui/impl/context/ElasticFacadeImpl.groovy +++ b/framework/src/main/groovy/org/moqui/impl/context/ElasticFacadeImpl.groovy @@ -196,7 +196,7 @@ class ElasticFacadeImpl implements ElasticFacade { requestFactory.init() // try connecting and get server info - int retries = clusterHost == 'localhost' && !"true".equals(System.getProperty("moqui.elasticsearch.started")) ? 1 : 20 + int retries = ((clusterHost == 'localhost' || clusterHost == '127.0.0.1') && !"true".equals(System.getProperty("moqui.elasticsearch.started"))) ? 1 : 20 for (int i = 1; i <= retries; i++) { try { serverInfo = getServerInfo() @@ -348,7 +348,7 @@ class ElasticFacadeImpl implements ElasticFacade { for (Map entry in actionSourceList) { // look for _index fields in each Map, if found prefix if (entry.size() == 1) { - Map actionMap = entry.values().first() + Map actionMap = (Map) entry.values().first() Object _indexVal = actionMap.get("_index") if (_indexVal != null && _indexVal instanceof String) actionMap.put("_index", prefixIndexName((String) _indexVal)) } @@ -493,8 +493,14 @@ class ElasticFacadeImpl implements ElasticFacade { // requires 2.4.0 or later response = makeRestClient(Method.POST, index, "_search/point_in_time", [keep_alive:keepAlive]).call() } else { - // see: https://www.elastic.co/guide/en/elasticsearch/reference/current/paginate-search-results.html#scroll-search-results - response = makeRestClient(Method.POST, index, "_pit", [keep_alive:keepAlive]).call() + // see: https://www.elastic.co/guide/en/elasticsearch/reference/7.10/paginate-search-results.html#scroll-search-results + // whatever the docs say: + // - it doesn't work with the keep_alive parameter at all "contains unrecognized parameter: [keep_alive]" + // - does not work with no body "request body is required" + // - and it doesn't work without the doc type _doc before _pit in the path "mapping type name [_pit] can't start with '_' unless it is called [_doc]" + // in other words, the docs are completely wrong for ES 7.10.2 + // response = makeRestClient(Method.POST, index, "_pit", [keep_alive:keepAlive]).call() + response = makeRestClient(Method.POST, index, "_doc/_pit", null).text(objectToJson([keep_alive:keepAlive])).call() } // System.out.println("Get PIT Response: ${response.statusCode} ${response.reasonPhrase}\n${response.text()}") checkResponse(response, "PIT", index) @@ -510,7 +516,7 @@ class ElasticFacadeImpl implements ElasticFacade { response = makeRestClient(Method.DELETE, null, "_search/point_in_time", null) .text(objectToJson([pit_id:[pitId]])).call() } else { - // see: https://www.elastic.co/guide/en/elasticsearch/reference/current/paginate-search-results.html#scroll-search-results + // see: https://www.elastic.co/guide/en/elasticsearch/reference/7.10/paginate-search-results.html#scroll-search-results response = makeRestClient(Method.DELETE, null, "_pit", null).text(objectToJson([id:pitId])).call() } // System.out.println("Delete PIT Response: ${response.statusCode} ${response.reasonPhrase}\n${response.text()}") @@ -689,13 +695,23 @@ class ElasticFacadeImpl implements ElasticFacade { if (index == null) return null index = index.trim() if (index.isEmpty()) return null - return indexPrefix != null && !index.startsWith(indexPrefix) ? indexPrefix.concat(index) : index + // handle comma separated index names + return index.split(",").collect({ + it = it.trim() + return indexPrefix != null && !it.startsWith(indexPrefix) ? indexPrefix.concat(it) : it + }).join(",") + // return indexPrefix != null && !index.startsWith(indexPrefix) ? indexPrefix.concat(index) : index } String unprefixIndexName(String index) { if (index == null) return null index = index.trim() if (index.isEmpty()) return null - return indexPrefix != null && index.startsWith(indexPrefix) ? index.substring(indexPrefix.length()) : index + // handle comma separated index names + return index.split(",").collect({ + it = it.trim() + return indexPrefix != null && it.startsWith(indexPrefix) ? it.substring(indexPrefix.length()) : it + }).join(",") + // return indexPrefix != null && index.startsWith(indexPrefix) ? index.substring(indexPrefix.length()) : index } } diff --git a/framework/src/main/groovy/org/moqui/impl/context/ExecutionContextFactoryImpl.groovy b/framework/src/main/groovy/org/moqui/impl/context/ExecutionContextFactoryImpl.groovy index f42d6f8f2..c1804561d 100644 --- a/framework/src/main/groovy/org/moqui/impl/context/ExecutionContextFactoryImpl.groovy +++ b/framework/src/main/groovy/org/moqui/impl/context/ExecutionContextFactoryImpl.groovy @@ -390,29 +390,29 @@ class ExecutionContextFactoryImpl implements ExecutionContextFactory { // set default System properties now that all is merged for (MNode defPropNode in baseConfigNode.children("default-property")) { String propName = defPropNode.attribute("name") + String isSecretAttr = defPropNode.attribute("is-secret") + boolean isSecret = !"false".equals(isSecretAttr) && + ("true".equals(isSecretAttr) || propName.contains("pass") || propName.contains("pw") || propName.contains("key")) if (System.getProperty(propName)) { - if (propName.contains("pass") || propName.contains("pw") || propName.contains("key")) { - logger.info("Found pw/key property ${propName}, not setting from env var or default") + if (isSecret) { + logger.info("Found secret property ${propName}, not setting from env var or default") } else { logger.info("Found property ${propName} with value [${System.getProperty(propName)}], not setting from env var or default") } - continue - } - if (System.getenv(propName) && !System.getProperty(propName)) { + } else if (System.getenv(propName)) { // make env vars available as Java System properties System.setProperty(propName, System.getenv(propName)) - if (propName.contains("pass") || propName.contains("pw") || propName.contains("key")) { - logger.info("Setting pw/key property ${propName} from env var") + if (isSecret) { + logger.info("Setting secret property ${propName} from env var") } else { logger.info("Setting property ${propName} from env var with value [${System.getProperty(propName)}]") } - } - if (!System.getProperty(propName) && !System.getenv(propName)) { + } else { String valueAttr = defPropNode.attribute("value") if (valueAttr != null && !valueAttr.isEmpty()) { System.setProperty(propName, SystemBinding.expand(valueAttr)) - if (propName.contains("pass") || propName.contains("pw") || propName.contains("key")) { - logger.info("Setting pw/key property ${propName} from default") + if (isSecret) { + logger.info("Setting secret property ${propName} from default") } else { logger.info("Setting property ${propName} from default with value [${System.getProperty(propName)}]") } @@ -452,10 +452,10 @@ class ExecutionContextFactoryImpl implements ExecutionContextFactory { BlockingQueue workQueue = new LinkedBlockingQueue<>(workerQueueSize) int coreSize = (toolsNode.attribute("worker-pool-core") ?: "16") as int - int maxSize = (toolsNode.attribute("worker-pool-max") ?: "24") as int - int availableProcessorsSize = Runtime.getRuntime().availableProcessors() * 2 + int maxSize = (toolsNode.attribute("worker-pool-max") ?: "32") as int + int availableProcessorsSize = Runtime.getRuntime().availableProcessors() * 3 if (availableProcessorsSize > maxSize) { - logger.info("Setting worker pool size to ${availableProcessorsSize} based on available processors * 2") + logger.info("Setting worker pool size to ${availableProcessorsSize} based on available processors * 3") maxSize = availableProcessorsSize } long aliveTime = (toolsNode.attribute("worker-pool-alive") ?: "60") as long @@ -467,9 +467,9 @@ class ExecutionContextFactoryImpl implements ExecutionContextFactory { boolean waitWorkerPoolEmpty(int retryLimit) { ThreadPoolExecutor jobWorkerPool = serviceFacade.jobWorkerPool int count = 0 - logger.warn("Wait for workerPool and jobWorkerPool empty: worker queue size ${workerPool.getQueue().size()} active ${workerPool.getActiveCount()}; service job queue size ${jobWorkerPool.getQueue().size()} active ${jobWorkerPool.getActiveCount()}") while (count < retryLimit && (workerPool.getQueue().size() > 0 || workerPool.getActiveCount() > 0 || jobWorkerPool.getQueue().size() > 0 || jobWorkerPool.getActiveCount() > 0)) { + if (count % 10 == 0) logger.warn("Wait for workerPool and jobWorkerPool empty: worker queue size ${workerPool.getQueue().size()} active ${workerPool.getActiveCount()} max threads ${workerPool.getMaximumPoolSize()}; service job queue size ${jobWorkerPool.getQueue().size()} active ${jobWorkerPool.getActiveCount()}") Thread.sleep(100) count++ } @@ -699,70 +699,112 @@ class ExecutionContextFactoryImpl implements ExecutionContextFactory { logger.info("Initialized ClassLoaders in ${System.currentTimeMillis() - startTime}ms") } - /** Called from MoquiContextListener.contextInitialized after ECFI init */ @Override boolean checkEmptyDb() { + /* NOTE: Called from Moqui.dynamicInit() after ECFI init (which is also called from MoquiContextListener.contextInitialized()) */ MNode toolsNode = confXmlRoot.first("tools") toolsNode.setSystemExpandAttributes(true) + + boolean needsRestartEcfi = false + boolean emptyDbLoadRan = false + + // if empty-db-load has a value and is not 'none' then load those String emptyDbLoad = toolsNode.attribute("empty-db-load") - if (!emptyDbLoad || emptyDbLoad == 'none') return false + if (emptyDbLoad && emptyDbLoad != 'none') { + long enumCount = getEntity().find("moqui.basic.Enumeration").disableAuthz().count() + if (enumCount == 0) { + logger.info("Found ${enumCount} Enumeration records, loading empty-db-load data types (${emptyDbLoad})") + + ExecutionContext ec = getExecutionContext() + try { + ec.getArtifactExecution().disableAuthz() + ec.getArtifactExecution().push("loadDataEmptyDb", ArtifactExecutionInfo.AT_OTHER, ArtifactExecutionInfo.AUTHZA_ALL, false) + ec.getArtifactExecution().setAnonymousAuthorizedAll() + ec.getUser().loginAnonymousIfNoUser() + + EntityDataLoader edl = ec.getEntity().makeDataLoader() + if (emptyDbLoad != 'all') edl.dataTypes(new HashSet(emptyDbLoad.split(",") as List)) + + try { + long startTime = System.currentTimeMillis() + long records = edl.load() - long enumCount = getEntity().find("moqui.basic.Enumeration").disableAuthz().count() - if (enumCount == 0) { - logger.info("Found ${enumCount} Enumeration records, loading empty-db-load data types (${emptyDbLoad})") + logger.info("Loaded [${records}] records (with types from empty-db-load: ${emptyDbLoad}) in ${(System.currentTimeMillis() - startTime)/1000} seconds.") + } catch (Throwable t) { + logger.error("Error loading empty DB data (with types: ${emptyDbLoad})", t) + } + + } finally { + ec.destroy() + } + + needsRestartEcfi = true + emptyDbLoadRan = true + } else { + logger.info("Found ${enumCount} Enumeration records, NOT loading empty-db-load data types (${emptyDbLoad})") + } + } + + // if on-start-load-types has a value and is not 'none' then load those + String onStartLoadTypes = toolsNode.attribute("on-start-load-types") + String onStartLoadComponents = toolsNode.attribute("on-start-load-components") + if (!emptyDbLoadRan && onStartLoadTypes && onStartLoadTypes != 'none') { + logger.info("Loading on-start-load-types data types [${onStartLoadTypes}] and components [${onStartLoadComponents ?: 'all'}]") ExecutionContext ec = getExecutionContext() try { ec.getArtifactExecution().disableAuthz() - ec.getArtifactExecution().push("loadData", ArtifactExecutionInfo.AT_OTHER, ArtifactExecutionInfo.AUTHZA_ALL, false) + ec.getArtifactExecution().push("loadDataOnStart", ArtifactExecutionInfo.AT_OTHER, ArtifactExecutionInfo.AUTHZA_ALL, false) ec.getArtifactExecution().setAnonymousAuthorizedAll() ec.getUser().loginAnonymousIfNoUser() EntityDataLoader edl = ec.getEntity().makeDataLoader() - if (emptyDbLoad != 'all') edl.dataTypes(new HashSet(emptyDbLoad.split(",") as List)) + if (onStartLoadTypes != 'all') edl.dataTypes(new HashSet(onStartLoadTypes.split(",") as List)) + if (onStartLoadComponents && onStartLoadComponents != 'all') edl.componentNameList(onStartLoadComponents.split(",") as List) try { long startTime = System.currentTimeMillis() long records = edl.load() - logger.info("Loaded [${records}] records (with types: ${emptyDbLoad}) in ${(System.currentTimeMillis() - startTime)/1000} seconds.") + logger.info("Loaded [${records}] records (with types from on-start-load-types: [${onStartLoadTypes}] components: [${onStartLoadComponents ?: 'all'}]) in ${(System.currentTimeMillis() - startTime)/1000} seconds.") } catch (Throwable t) { - logger.error("Error loading empty DB data (with types: ${emptyDbLoad})", t) + logger.error("Error loading on-start DB data (with types: [${onStartLoadTypes}] components: [${onStartLoadComponents ?: 'all'}])", t) } } finally { ec.destroy() } - return true - } else { - logger.info("Found ${enumCount} Enumeration records, NOT loading empty-db-load data types (${emptyDbLoad})") - // if this instance_purpose is test load type 'test' data - if ("test".equals(System.getProperty("instance_purpose"))) { - logger.warn("Loading 'test' type data (instance_purpose=test)") - ExecutionContext ec = getExecutionContext() - try { - ec.getArtifactExecution().disableAuthz() - ec.getArtifactExecution().push("loadData", ArtifactExecutionInfo.AT_OTHER, ArtifactExecutionInfo.AUTHZA_ALL, false) - ec.getArtifactExecution().setAnonymousAuthorizedAll() - ec.getUser().loginAnonymousIfNoUser() - EntityDataLoader edl = ec.getEntity().makeDataLoader() - edl.dataTypes(new HashSet(['test'])) + needsRestartEcfi = true + } - try { - long startTime = System.currentTimeMillis() - long records = edl.load() + // if this instance_purpose is test load type 'test' data + if ("test".equals(System.getProperty("instance_purpose"))) { + logger.warn("Loading 'test' type data (because instance_purpose=test)") + ExecutionContext ec = getExecutionContext() + try { + ec.getArtifactExecution().disableAuthz() + ec.getArtifactExecution().push("loadDataTest", ArtifactExecutionInfo.AT_OTHER, ArtifactExecutionInfo.AUTHZA_ALL, false) + ec.getArtifactExecution().setAnonymousAuthorizedAll() + ec.getUser().loginAnonymousIfNoUser() - logger.info("Loaded [${records}] records (with type test) in ${(System.currentTimeMillis() - startTime)/1000} seconds.") - } catch (Throwable t) { - logger.error("Error loading empty DB data (with type test)", t) - } + EntityDataLoader edl = ec.getEntity().makeDataLoader() + edl.dataTypes(new HashSet(['test'])) - } finally { - ec.destroy() + try { + long startTime = System.currentTimeMillis() + long records = edl.load() + + logger.info("Loaded [${records}] records (with type test) in ${(System.currentTimeMillis() - startTime)/1000} seconds.") + } catch (Throwable t) { + logger.error("Error loading empty DB data (with type test)", t) } + + } finally { + ec.destroy() } - return false } + + return needsRestartEcfi } @Override void destroy() { diff --git a/framework/src/main/groovy/org/moqui/impl/context/L10nFacadeImpl.java b/framework/src/main/groovy/org/moqui/impl/context/L10nFacadeImpl.java index 65b4511b9..b273984b5 100644 --- a/framework/src/main/groovy/org/moqui/impl/context/L10nFacadeImpl.java +++ b/framework/src/main/groovy/org/moqui/impl/context/L10nFacadeImpl.java @@ -15,6 +15,7 @@ import org.moqui.BaseArtifactException; import org.moqui.context.L10nFacade; +import org.moqui.entity.EntityCondition; import org.moqui.entity.EntityValue; import org.moqui.entity.EntityFind; @@ -23,6 +24,8 @@ import javax.xml.bind.DatatypeConverter; import java.math.BigDecimal; import java.math.RoundingMode; +import java.text.DecimalFormat; +import java.text.DecimalFormatSymbols; import java.text.NumberFormat; import java.sql.Date; import java.sql.Time; @@ -100,13 +103,14 @@ public String localize(String original, Locale locale) { } @Override - public String formatCurrency(Object amount, String uomId) { return formatCurrency(amount, uomId, null, getLocale()); } + public String formatCurrencyNoSymbol(Object amount, String uomId) { return formatCurrency(amount, uomId, null, getLocale(), true); } @Override - public String formatCurrency(Object amount, String uomId, Integer fractionDigits) { - return formatCurrency(amount, uomId, fractionDigits, getLocale()); - } + public String formatCurrency(Object amount, String uomId) { return formatCurrency(amount, uomId, null, getLocale(), false); } + @Override + public String formatCurrency(Object amount, String uomId, Integer fractionDigits) { return formatCurrency(amount, uomId, fractionDigits, getLocale(), false); } @Override - public String formatCurrency(Object amount, String uomId, Integer fractionDigits, Locale locale) { + public String formatCurrency(Object amount, String uomId, Integer fractionDigits, Locale locale) { return formatCurrency(amount, uomId, fractionDigits, locale, false); } + public String formatCurrency(Object amount, String uomId, Integer fractionDigits, Locale locale, boolean hideSymbol) { if (amount == null) return ""; if (amount instanceof CharSequence) { if (((CharSequence) amount).length() == 0) { @@ -116,22 +120,54 @@ public String formatCurrency(Object amount, String uomId, Integer fractionDigits } } - Currency currency = uomId != null && uomId.length() > 0 ? Currency.getInstance(uomId) : null; if (locale == null) locale = getLocale(); - if (currency != null) { - NumberFormat nf = NumberFormat.getCurrencyInstance(locale); - nf.setCurrency(currency); - if (fractionDigits == null) fractionDigits = currency.getDefaultFractionDigits(); - nf.setMaximumFractionDigits(fractionDigits); - nf.setMinimumFractionDigits(fractionDigits); - return nf.format(amount); - } else { - NumberFormat nf = NumberFormat.getInstance(); - if (fractionDigits == null) fractionDigits = 2; - nf.setMaximumFractionDigits(fractionDigits); - nf.setMinimumFractionDigits(fractionDigits); - return nf.format(amount); + NumberFormat nf = NumberFormat.getCurrencyInstance(locale); + String currencySymbol = null; + if (hideSymbol) + currencySymbol = ""; + EntityValue uom = null; + if (uomId != null && uomId.length() > 0) { + List uomList = eci.getEntity().find("moqui.basic.Uom").condition("uomId", uomId) + .condition("uomTypeEnumId", "UT_CURRENCY_MEASURE").disableAuthz().list(); + if (uomList.size() > 0) { + uom = uomList.get(0); + String symbol = uom.getString("symbol"); + if (currencySymbol == null && symbol != null) + currencySymbol = symbol; + Object fractionDigitsField = uom.get("fractionDigits"); + if (fractionDigits == null && fractionDigitsField != null) { + if (fractionDigitsField instanceof Integer) + fractionDigits = (Integer)fractionDigitsField; + else if (fractionDigitsField instanceof Long) + fractionDigits = ((Long)fractionDigitsField).intValue(); + } + } + } + + Currency currency = null; + if (uomId != null && uomId.length() > 0) { + try { + currency = Currency.getInstance(uomId); + if (currencySymbol == null) + currencySymbol = currency.getSymbol(); + if (fractionDigits == null) + fractionDigits = currency.getDefaultFractionDigits(); + } catch (Exception e) { + if (logger.isTraceEnabled()) logger.trace("Ignoring IllegalArgumentException for Currency parse: " + e.toString()); + } } + if (currencySymbol == null) + currencySymbol = ""; + + if (fractionDigits == null) + fractionDigits = 2; + nf.setMaximumFractionDigits(fractionDigits); + nf.setMinimumFractionDigits(fractionDigits); + DecimalFormatSymbols dfSymbols = new DecimalFormatSymbols(locale); + dfSymbols.setCurrencySymbol(currencySymbol); + ((DecimalFormat)nf).setDecimalFormatSymbols(dfSymbols); + + return nf.format(amount); } @Override @@ -144,10 +180,29 @@ public BigDecimal roundCurrency(BigDecimal amount, String uomId, boolean precise } @Override public BigDecimal roundCurrency(BigDecimal amount, String uomId, boolean precise, RoundingMode mode) { - Currency currency = Currency.getInstance(uomId); - int nDigits = currency.getDefaultFractionDigits(); - if (precise) nDigits++; - return amount.setScale(nDigits, mode); + if (amount == null) + return null; + List uomList = eci.getEntity().find("moqui.basic.Uom").condition("uomId", uomId).condition("uomTypeEnumId", "UT_CURRENCY_MEASURE").list(); + Integer fractionDigits = null; + if (uomList.size() > 0) { + Object fractionDigitsField = uomList.get(0).get("fractionDigits"); + if (fractionDigitsField != null) { + if (fractionDigitsField instanceof Integer) + fractionDigits = (Integer)fractionDigitsField; + else if (fractionDigitsField instanceof Long) + fractionDigits = ((Long)fractionDigitsField).intValue(); + } + } + if (fractionDigits == null) { + Currency currency = Currency.getInstance(uomId); + fractionDigits = currency.getDefaultFractionDigits(); + } + if (fractionDigits == null) { + fractionDigits = 2; + } + if (precise) fractionDigits++; + eci.getLogger().info("Rounding to " + fractionDigits + " digits."); + return amount.setScale(fractionDigits, mode); } @Override diff --git a/framework/src/main/groovy/org/moqui/impl/context/NotificationMessageImpl.groovy b/framework/src/main/groovy/org/moqui/impl/context/NotificationMessageImpl.groovy index b30563d2d..05433737b 100644 --- a/framework/src/main/groovy/org/moqui/impl/context/NotificationMessageImpl.groovy +++ b/framework/src/main/groovy/org/moqui/impl/context/NotificationMessageImpl.groovy @@ -20,10 +20,8 @@ import org.moqui.BaseArtifactException import org.moqui.Moqui import org.moqui.context.ExecutionContext import org.moqui.context.NotificationMessage -import org.moqui.context.NotificationMessage.NotificationType import org.moqui.entity.EntityFacade import org.moqui.entity.EntityList -import org.moqui.entity.EntityListIterator import org.moqui.entity.EntityValue import org.slf4j.Logger import org.slf4j.LoggerFactory @@ -37,6 +35,7 @@ class NotificationMessageImpl implements NotificationMessage, Externalizable { private Set userIdSet = new HashSet() private String userGroupId = (String) null private String topic = (String) null + private String subTopic = (String) null private transient EntityValue notificationTopic = (EntityValue) null private String messageJson = (String) null private transient Map messageMap = (Map) null @@ -92,16 +91,17 @@ class NotificationMessageImpl implements NotificationMessage, Externalizable { // notify by group, skipping users already notified if (userGroupId) { - EntityListIterator eli = ef.find("moqui.security.UserGroupMember") + ef.find("moqui.security.UserGroupMember") .conditionDate("fromDate", "thruDate", new Timestamp(System.currentTimeMillis())) - .condition("userGroupId", userGroupId).disableAuthz().iterator() - EntityValue nextValue - while ((nextValue = (EntityValue) eli.next()) != null) { - String userId = (String) nextValue.userId - if (checkedUserIds.contains(userId)) continue - checkedUserIds.add(userId) - if (checkUserNotify(userId, ef)) notifyUserIds.add(userId) - } + .condition("userGroupId", userGroupId).disableAuthz().iterator().withCloseable ({eli -> + EntityValue nextValue + while ((nextValue = (EntityValue) eli.next()) != null) { + String userId = (String) nextValue.userId + if (checkedUserIds.contains(userId)) continue + checkedUserIds.add(userId) + if (checkUserNotify(userId, ef)) notifyUserIds.add(userId) + } + }) } // add all users subscribed to all messages on the topic @@ -144,6 +144,9 @@ class NotificationMessageImpl implements NotificationMessage, Externalizable { @Override NotificationMessage topic(String topic) { this.topic = topic; notificationTopic = null; return this } @Override String getTopic() { topic } + @Override String getSubTopic() { subTopic } + @Override NotificationMessage subTopic(String st) { subTopic = st; return this } + @Override NotificationMessage message(String messageJson) { this.messageJson = messageJson; messageMap = null; return this } @Override NotificationMessage message(Map message) { this.messageMap = Collections.unmodifiableMap(message) as Map @@ -169,16 +172,18 @@ class NotificationMessageImpl implements NotificationMessage, Externalizable { @Override NotificationMessage title(String title) { titleTemplate = title; return this } @Override String getTitle() { if (titleText == null) { - EntityValue localNotTopic = getNotificationTopic() - if (localNotTopic != null) { - if (type == danger && localNotTopic.errorTitleTemplate) { - titleText = ecfi.resource.expand((String) localNotTopic.errorTitleTemplate, "", getMessageMap(), true) - } else if (localNotTopic.titleTemplate) { - titleText = ecfi.resource.expand((String) localNotTopic.titleTemplate, "", getMessageMap(), true) + if (titleTemplate != null && !titleTemplate.isEmpty()) + titleText = ecfi.resource.expand(titleTemplate, "", getMessageMap(), true) + if (titleText == null || titleText.isEmpty()) { + EntityValue localNotTopic = getNotificationTopic() + if (localNotTopic != null) { + if (type == danger && localNotTopic.errorTitleTemplate) { + titleText = ecfi.resource.expand((String) localNotTopic.errorTitleTemplate, "", getMessageMap(), true) + } else if (localNotTopic.titleTemplate) { + titleText = ecfi.resource.expand((String) localNotTopic.titleTemplate, "", getMessageMap(), true) + } } } - if ((titleText == null || titleText.isEmpty()) && titleTemplate != null && !titleTemplate.isEmpty()) - titleText = ecfi.resource.expand(titleTemplate, "", getMessageMap(), true) } return titleText } @@ -305,7 +310,7 @@ class NotificationMessageImpl implements NotificationMessage, Externalizable { boolean beganTransaction = tfi.begin(null) try { Map createResult = ecfi.service.sync().name("create", "moqui.security.user.NotificationMessage") - .parameters([topic:this.topic, userGroupId:this.userGroupId, sentDate:this.sentDate, + .parameters([topic:this.topic, subTopic:this.subTopic, userGroupId:this.userGroupId, sentDate:this.sentDate, messageJson:this.getMessageJson(), titleText:this.getTitle(), linkText:this.getLink(), typeString:this.getType(), showAlert:(this.showAlert ? 'Y' : 'N')]) .disableAuthz().call() @@ -450,7 +455,7 @@ class NotificationMessageImpl implements NotificationMessage, Externalizable { @Override Map getWrappedMessageMap() { EntityValue localNotTopic = getNotificationTopic() - return [topic:topic, sentDate:sentDate, notificationMessageId:notificationMessageId, topicDescription:localNotTopic?.description, + return [topic:topic, subTopic:subTopic, sentDate:sentDate, notificationMessageId:notificationMessageId, topicDescription:localNotTopic?.description, message:getMessageMap(), title:getTitle(), link:getLink(), type:getType(), persistOnSend:isPersistOnSend(), showAlert:isShowAlert(), alertNoAutoHide:isAlertNoAutoHide()] } @@ -467,6 +472,7 @@ class NotificationMessageImpl implements NotificationMessage, Externalizable { void populateFromValue(EntityValue nmbu) { this.notificationMessageId = nmbu.notificationMessageId this.topic = nmbu.topic + this.subTopic = nmbu.subTopic this.sentDate = nmbu.getTimestamp("sentDate") this.userGroupId = nmbu.userGroupId this.messageJson = nmbu.messageJson @@ -486,6 +492,7 @@ class NotificationMessageImpl implements NotificationMessage, Externalizable { out.writeObject(userIdSet) out.writeObject(userGroupId) out.writeUTF(topic) + out.writeObject(subTopic) out.writeUTF(getMessageJson()) out.writeObject(notificationMessageId) out.writeObject(sentDate) @@ -500,6 +507,7 @@ class NotificationMessageImpl implements NotificationMessage, Externalizable { userIdSet = (Set) objectInput.readObject() userGroupId = (String) objectInput.readObject() topic = objectInput.readUTF() + subTopic = objectInput.readObject() messageJson = objectInput.readUTF() notificationMessageId = (String) objectInput.readObject() sentDate = (Timestamp) objectInput.readObject() diff --git a/framework/src/main/groovy/org/moqui/impl/context/UserFacadeImpl.groovy b/framework/src/main/groovy/org/moqui/impl/context/UserFacadeImpl.groovy index 0a37ad7aa..abd9f7b23 100644 --- a/framework/src/main/groovy/org/moqui/impl/context/UserFacadeImpl.groovy +++ b/framework/src/main/groovy/org/moqui/impl/context/UserFacadeImpl.groovy @@ -113,8 +113,10 @@ class UserFacadeImpl implements UserFacade { // user found in session so no login needed, but make sure hasLoggedOut != "Y" EntityValue userAccount = (EntityValue) null if (sesUsername != null && !sesUsername.isEmpty()) { + EntityCondition usernameCond = eci.entityFacade.getConditionFactory() + .makeCondition("username", EntityCondition.ComparisonOperator.EQUALS, username).ignoreCase() userAccount = eci.getEntity().find("moqui.security.UserAccount") - .condition("username", sesUsername).useCache(false).disableAuthz().one() + .condition(usernameCond).useCache(false).disableAuthz().one() } if (userAccount != null && "Y".equals(userAccount.getNoCheckSimple("hasLoggedOut"))) { @@ -1051,8 +1053,10 @@ class UserFacadeImpl implements UserFacade { EntityValueBase ua = (EntityValueBase) null if (username != null && username.length() > 0) { + EntityCondition usernameCond = ufi.eci.entityFacade.getConditionFactory() + .makeCondition("username", EntityCondition.ComparisonOperator.EQUALS, username).ignoreCase() ua = (EntityValueBase) ufi.eci.getEntity().find("moqui.security.UserAccount") - .condition("username", username).useCache(false).disableAuthz().one() + .condition(usernameCond).useCache(false).disableAuthz().one() } if (ua != null) { userAccount = ua diff --git a/framework/src/main/groovy/org/moqui/impl/context/WebFacadeImpl.groovy b/framework/src/main/groovy/org/moqui/impl/context/WebFacadeImpl.groovy index 8fdf52158..a1b4e4b1d 100644 --- a/framework/src/main/groovy/org/moqui/impl/context/WebFacadeImpl.groovy +++ b/framework/src/main/groovy/org/moqui/impl/context/WebFacadeImpl.groovy @@ -49,6 +49,8 @@ import javax.servlet.ServletContext import javax.servlet.http.HttpServletRequest import javax.servlet.http.HttpServletResponse import javax.servlet.http.HttpSession +import java.nio.charset.StandardCharsets +import java.sql.Timestamp /** This class is a facade to easily get information from and about the web context. */ @CompileStatic @@ -1210,6 +1212,70 @@ class WebFacadeImpl implements WebFacade { return } + // login anonymous if not logged in + eci.userFacade.loginAnonymousIfNoUser() + } else if ("SmatHmacSha256Timestamp".equals(messageAuthEnumId)) { + // validate HMAC value from authHeaderName HTTP header using sharedSecret and messageText + String authHeaderName = (String) systemMessageRemote.authHeaderName + String sharedSecret = (String) systemMessageRemote.sharedSecret + + String headerValue = request.getHeader(authHeaderName) + if (!headerValue) { + logger.warn("System message receive HMAC verify no header ${authHeaderName} value found, for remote ${systemMessageRemoteId}") + response.sendError(HttpServletResponse.SC_FORBIDDEN, "No HMAC header ${authHeaderName} found for remote system ${systemMessageRemoteId}") + return + } + + // This assumes a header format like + // Example-Signature-Header: + //t=1492774577, + //v1=5257a869e7ecebeda32affa62cdca3fa51cad7e77a0e56ff536d0ce8e108d8bd + // We’ve added newlines for clarity, but a realExample-Signature-Header is on a single line. + String timestamp = null; + String incomingSignature = null; + String[] headerValueList = headerValue.split(",") // split on comma + for (String headerValueItem : headerValueList) { + String key = headerValueItem.split("=")[0].trim() + if ("t".equals(key)) + timestamp = headerValueItem.split("=")[1].trim() + else if ("v1".equals(key)) + incomingSignature = headerValueItem.split("=")[1].trim() + } + + // This also assumes that the signature is generated from the following concatenated strings: + // Timestamp in the header + // The character . + // The text body of the request + String signatureTextToVerify = timestamp + "." + messageText + + Mac hmac = Mac.getInstance("HmacSHA256") + hmac.init(new SecretKeySpec(sharedSecret.getBytes(StandardCharsets.UTF_8), "HmacSHA256")) + // NOTE: if this fails try with "ISO-8859-1" + byte[] hash = hmac.doFinal(signatureTextToVerify.getBytes(StandardCharsets.UTF_8)); + String signature = "" + for (byte b : hash) { + // Came from https://github.com/stripe/stripe-java/blob/3686feb8f2067878b7bb4619f931580a3d31bf4f/src/main/java/com/stripe/net/Webhook.java#L187 + signature += Integer.toString((b & 0xff) + 0x100, 16).substring(1); + } + + if (incomingSignature != signature) { + logger.warn("System message receive HMAC verify header value ${incomingSignature} calculated ${signature} did not match for remote ${systemMessageRemoteId}") + response.sendError(HttpServletResponse.SC_FORBIDDEN, "HMAC verify failed for remote system ${systemMessageRemoteId}") + return + } + + Timestamp incomingTimestamp = new Timestamp(Long.parseLong(timestamp) * 1000) + + // Add 10 seconds to now timestamp to allow for clock skew (10 seconds = 10000 milliseconds = 10*1000) + Timestamp nowTimestamp = new Timestamp(eci.user.nowTimestamp.getTime() + 10000) + // If timestamp was not sent in past 5 minutes, reject message (5 minutes = 300000 milliseconds = 5*60*1000) + Timestamp beforeTimestamp = new Timestamp(nowTimestamp.getTime() - 300000) + if (!incomingTimestamp.before(nowTimestamp) || !incomingTimestamp.after(beforeTimestamp) ){ + logger.warn("System message receive HMAC invalid incoming timestamp where before timestamp ${beforeTimestamp} < incoming timestamp ${incomingTimestamp} < now timestamp ${nowTimestamp}" ) + response.sendError(HttpServletResponse.SC_FORBIDDEN, "HMAC timestamp verification failed") + return + } + // login anonymous if not logged in eci.userFacade.loginAnonymousIfNoUser() } else if (!"SmatNone".equals(messageAuthEnumId)) { diff --git a/framework/src/main/groovy/org/moqui/impl/entity/EntityConditionFactoryImpl.groovy b/framework/src/main/groovy/org/moqui/impl/entity/EntityConditionFactoryImpl.groovy index 281f164fe..b41c8605c 100644 --- a/framework/src/main/groovy/org/moqui/impl/entity/EntityConditionFactoryImpl.groovy +++ b/framework/src/main/groovy/org/moqui/impl/entity/EntityConditionFactoryImpl.groovy @@ -99,6 +99,12 @@ class EntityConditionFactoryImpl implements EntityConditionFactory { EntityCondition makeConditionToField(String fieldName, ComparisonOperator operator, String toFieldName) { return new FieldToFieldCondition(new ConditionField(fieldName), operator, new ConditionField(toFieldName)) } + @Override + EntityCondition makeConditionToField(String fieldName, ComparisonOperator operator, String toFieldName, Object value) { + // TODO: This needs some work + return null + // return new FieldToFieldCondition(new ConditionField(fieldName), operator, new ConditionField(toFieldName)) + } @Override EntityCondition makeCondition(List conditionList) { @@ -316,6 +322,9 @@ class EntityConditionFactoryImpl implements EntityConditionFactory { case "ENTCO_NOT_BETWEEN": return EntityCondition.NOT_BETWEEN case "ENTCO_LIKE": return EntityCondition.LIKE case "ENTCO_NOT_LIKE": return EntityCondition.NOT_LIKE + case "ENTCO_BEGINS_FIELD": return EntityCondition.BEGINS_FIELD + case "ENTCO_ENDS_FIELD": return EntityCondition.ENDS_FIELD + case "ENTCO_CONTAINS_FIELD": return EntityCondition.CONTAINS_FIELD case "ENTCO_IS_NULL": return EntityCondition.IS_NULL case "ENTCO_IS_NOT_NULL": return EntityCondition.IS_NOT_NULL default: return null @@ -361,8 +370,27 @@ class EntityConditionFactoryImpl implements EntityConditionFactory { if (efi.ecfi.resourceFacade.condition(ignore, null)) return null if (toFieldName != null && toFieldName.length() > 0) { - EntityCondition ec = makeConditionToField(fieldName, getComparisonOperator(operator), toFieldName) + EntityCondition ec = null; + ComparisonOperator compOp = getComparisonOperator(operator) + if (compOp == ComparisonOperator.BEGINS_FIELD || compOp == ComparisonOperator.ENDS_FIELD || compOp == ComparisonOperator.CONTAINS_FIELD) { + Object condValue + if (value != null && value.length() > 0) { + // NOTE: have to convert value (if needed) later on because we don't know which entity/field this is for, or change to pass in entity? + condValue = value + } else { + condValue = fromObj + } + logger.warn(value) + logger.warn(fromObj.toString()) + logger.warn(condValue.toString()) + + // TODO: This also needs some work! + ec = makeConditionWhere("'moc.elgoog.fdsa' LIKE prefix || '%'") + } else { + ec = makeConditionToField(fieldName, compOp, toFieldName) + } if (ignoreCase) ec.ignoreCase() + return ec } else { Object condValue @@ -493,6 +521,15 @@ class EntityConditionFactoryImpl implements EntityConditionFactory { "not-like":ComparisonOperator.NOT_LIKE, "NOT LIKE":ComparisonOperator.NOT_LIKE, + "begins-field":ComparisonOperator.BEGINS_FIELD, + "BEGINS FIELD":ComparisonOperator.BEGINS_FIELD, + + "ends-field":ComparisonOperator.ENDS_FIELD, + "ENDS FIELD":ComparisonOperator.ENDS_FIELD, + + "contains-field":ComparisonOperator.CONTAINS_FIELD, + "CONTAINS FIELD":ComparisonOperator.CONTAINS_FIELD, + "is-null":ComparisonOperator.IS_NULL, "IS NULL":ComparisonOperator.IS_NULL, diff --git a/framework/src/main/groovy/org/moqui/impl/entity/EntityDataDocument.groovy b/framework/src/main/groovy/org/moqui/impl/entity/EntityDataDocument.groovy index a5be31a91..66bcaab30 100644 --- a/framework/src/main/groovy/org/moqui/impl/entity/EntityDataDocument.groovy +++ b/framework/src/main/groovy/org/moqui/impl/entity/EntityDataDocument.groovy @@ -301,8 +301,7 @@ class EntityDataDocument { // do the one big query String lastDocId = null int docCount = 0 - EntityListIterator mainEli = mainFind.iterator() - try { + try (EntityListIterator mainEli = mainFind.iterator()) { logger.info("Feed dataDocumentId ${dataDocumentId} query complete (cursor opened) in ${System.currentTimeMillis() - startTimeMillis}ms") EntityValue ev while ((ev = (EntityValue) mainEli.next()) != null) { @@ -345,7 +344,6 @@ class EntityDataDocument { efi.ecfi.serviceFacade.sync().name(feedReceiveServiceName).parameter("documentList", documentMapList).call() } } finally { - mainEli.close() logger.info("Feed dataDocumentId ${dataDocumentId} feed complete and cursor closed in ${System.currentTimeMillis() - startTimeMillis}ms") } @@ -366,16 +364,14 @@ class EntityDataDocument { ArrayList documentMapList = ddi.hasAllPrimaryPks ? null : new ArrayList() // do the one big query - EntityListIterator mainEli = mainFind.iterator() - try { + + mainFind.iterator().withCloseable ({mainEli-> EntityValue ev while ((ev = (EntityValue) mainEli.next()) != null) { // logger.warn("=========== DataDocument query result for ${dataDocumentId}: ${ev}") mergeValueToDocMap(ev, ddi, documentMapMap, documentMapList, docTsString) } - } finally { - mainEli.close() - } + }) // make the actual list and return it if (ddi.hasAllPrimaryPks) { diff --git a/framework/src/main/groovy/org/moqui/impl/entity/EntityDataFeed.groovy b/framework/src/main/groovy/org/moqui/impl/entity/EntityDataFeed.groovy index 02e5e8d41..c46a00ca9 100644 --- a/framework/src/main/groovy/org/moqui/impl/entity/EntityDataFeed.groovy +++ b/framework/src/main/groovy/org/moqui/impl/entity/EntityDataFeed.groovy @@ -633,7 +633,7 @@ class EntityDataFeed { // entity name alone String currentRelName = backwardRelList.get(i) String currentRelEntityName = currentRelName.contains("#") ? - currentRelName.substring(0, currentRelName.indexOf("#")) : + currentRelName.substring(currentRelName.indexOf("#") + 1) : currentRelName // all values should be for the same entity, so just use the first EntityDefinition prevRelValueEd = prevRelValueList.get(0).getEntityDefinition() diff --git a/framework/src/main/groovy/org/moqui/impl/entity/EntityDataWriterImpl.groovy b/framework/src/main/groovy/org/moqui/impl/entity/EntityDataWriterImpl.groovy index e3872d702..0dee3676c 100644 --- a/framework/src/main/groovy/org/moqui/impl/entity/EntityDataWriterImpl.groovy +++ b/framework/src/main/groovy/org/moqui/impl/entity/EntityDataWriterImpl.groovy @@ -14,6 +14,7 @@ package org.moqui.impl.entity import groovy.json.JsonBuilder +import groovy.transform.CompileStatic import org.moqui.entity.EntityValue import org.moqui.util.ObjectUtilities @@ -35,6 +36,7 @@ import java.time.format.DateTimeFormatter import java.util.zip.ZipEntry import java.util.zip.ZipOutputStream +@CompileStatic class EntityDataWriterImpl implements EntityDataWriter { private final static Logger logger = LoggerFactory.getLogger(EntityDataWriterImpl.class) @@ -170,9 +172,9 @@ class EntityDataWriterImpl implements EntityDataWriter { EntityDefinition ed = efi.getEntityDefinition(en) boolean useMaster = masterName != null && masterName.length() > 0 && ed.getMasterDefinition(masterName) != null EntityFind ef = makeEntityFind(en) - EntityListIterator eli = ef.iterator() - try { + + try (EntityListIterator eli = ef.iterator()) { if (!eli.hasNext()) continue String filename = path + '/' + en + '.' + fileType.name().toLowerCase() @@ -200,8 +202,6 @@ class EntityDataWriterImpl implements EntityDataWriter { } finally { pw.close() } - } finally { - eli.close() } } } catch (Throwable t) { @@ -248,8 +248,7 @@ class EntityDataWriterImpl implements EntityDataWriter { EntityDefinition ed = efi.getEntityDefinition(en) boolean useMaster = masterName != null && masterName.length() > 0 && ed.getMasterDefinition(masterName) != null EntityFind ef = makeEntityFind(en) - EntityListIterator eli = ef.iterator() - try { + try (EntityListIterator eli = ef.iterator()) { if (!eli.hasNext()) continue String filenameBase = tableColumnNames ? ed.getTableName() : en @@ -274,8 +273,6 @@ class EntityDataWriterImpl implements EntityDataWriter { } finally { out.closeEntry() } - } finally { - eli.close() } } } finally { @@ -289,7 +286,13 @@ class EntityDataWriterImpl implements EntityDataWriter { int writer(Writer writer) { if (dependentLevels > 0) efi.createAllAutoReverseManyRelationships() - LinkedHashSet activeEntityNames = skipEntityNames.size() > 0 ? entityNames - skipEntityNames : entityNames + LinkedHashSet activeEntityNames + if (skipEntityNames.size() == 0) { + activeEntityNames = entityNames + } else { + activeEntityNames = new LinkedHashSet<>(entityNames) + activeEntityNames.removeAll(skipEntityNames) + } EntityDefinition singleEd = null if (activeEntityNames.size() == 1) singleEd = efi.getEntityDefinition(activeEntityNames.first()) @@ -305,15 +308,11 @@ class EntityDataWriterImpl implements EntityDataWriter { for (String en in activeEntityNames) { EntityDefinition ed = efi.getEntityDefinition(en) boolean useMaster = masterName != null && masterName.length() > 0 && ed.getMasterDefinition(masterName) != null - EntityFind ef = makeEntityFind(en) - EntityListIterator eli = ef.iterator() - try { + try (EntityListIterator eli = makeEntityFind(en).iterator()) { EntityValue ev while ((ev = eli.next()) != null) { valuesWritten+= writeValue(ev, writer, useMaster) } - } finally { - eli.close() } } diff --git a/framework/src/main/groovy/org/moqui/impl/entity/EntityDbMeta.groovy b/framework/src/main/groovy/org/moqui/impl/entity/EntityDbMeta.groovy index da1e61d50..4a4158ab6 100644 --- a/framework/src/main/groovy/org/moqui/impl/entity/EntityDbMeta.groovy +++ b/framework/src/main/groovy/org/moqui/impl/entity/EntityDbMeta.groovy @@ -107,7 +107,7 @@ class EntityDbMeta { String schemaName = datasourceNode != null ? datasourceNode.attribute("schema-name") : null Set groupEntityNames = efi.getAllEntityNamesInGroup(groupName) - String[] types = ["TABLE", "VIEW", "ALIAS", "SYNONYM"] + String[] types = ["TABLE", "VIEW", "ALIAS", "SYNONYM", "PARTITIONED TABLE"] Set existingTableNames = new HashSet<>() boolean beganTx = useTxForMetaData ? efi.ecfi.transactionFacade.begin(300) : false @@ -455,10 +455,15 @@ class EntityDbMeta { ResultSet tableSet2 = null boolean beganTx = useTxForMetaData ? efi.ecfi.transactionFacade.begin(5) : false try { - con = efi.getConnection(groupName) + try { + con = efi.getConnection(groupName) + } catch (EntityException ee) { + logger.warn("Could not get connection so treating entity ${ed.fullEntityName} in group ${groupName} as table does not exist: ${ee.toString()}") + return false + } DatabaseMetaData dbData = con.getMetaData() - String[] types = ["TABLE", "VIEW", "ALIAS", "SYNONYM"] + String[] types = ["TABLE", "VIEW", "ALIAS", "SYNONYM", "PARTITIONED TABLE"] tableSet1 = dbData.getTables(con.getCatalog(), ed.getSchemaName(), ed.getTableName(), types) if (tableSet1.next()) { dbResult = true diff --git a/framework/src/main/groovy/org/moqui/impl/entity/EntityFindBase.groovy b/framework/src/main/groovy/org/moqui/impl/entity/EntityFindBase.groovy index 3d5d25c5f..31d015a3c 100644 --- a/framework/src/main/groovy/org/moqui/impl/entity/EntityFindBase.groovy +++ b/framework/src/main/groovy/org/moqui/impl/entity/EntityFindBase.groovy @@ -1149,19 +1149,18 @@ abstract class EntityFindBase implements EntityFind { } // call the abstract method - EntityListIterator eli - try { eli = iteratorExtended(queryWhereCondition, havingCondition, orderByExpanded, fieldInfoArray, fieldOptionsArray) } + try (EntityListIterator eli = iteratorExtended(queryWhereCondition, havingCondition, orderByExpanded, fieldInfoArray, fieldOptionsArray)) { + MNode databaseNode = this.efi.getDatabaseNode(ed.getEntityGroupName()) + if (limit != null && databaseNode != null && "cursor".equals(databaseNode.attribute("offset-style"))) { + el = (EntityListImpl) eli.getPartialList(offset != null ? offset : 0, limit, false) + } else { + el = (EntityListImpl) eli.getCompleteList(false); + } + } catch (SQLException e) { throw new EntitySqlException(makeErrorMsg("Error finding list of", LIST_ERROR, queryWhereCondition, ed, ec), e) } catch (ArtifactAuthorizationException e) { throw e } catch (Exception e) { throw new EntityException(makeErrorMsg("Error finding list of", LIST_ERROR, queryWhereCondition, ed, ec), e) } - MNode databaseNode = this.efi.getDatabaseNode(ed.getEntityGroupName()) - if (limit != null && databaseNode != null && "cursor".equals(databaseNode.attribute("offset-style"))) { - el = (EntityListImpl) eli.getPartialList(offset != null ? offset : 0, limit, true) - } else { - el = (EntityListImpl) eli.getCompleteList(true) - } - // register lock after because we can't before, don't know which records will be returned if (forUpdate && !isViewEntity && efi.ecfi.transactionFacade.getUseLockTrack()) { int elSize = el.size() @@ -1446,9 +1445,7 @@ abstract class EntityFindBase implements EntityFind { this.useCache(false) long totalUpdated = 0 - EntityListIterator eli = (EntityListIterator) null - try { - eli = iterator() + iterator().withCloseable ({eli -> EntityValue value while ((value = eli.next()) != null) { value.putAll(fieldsToSet) @@ -1458,9 +1455,7 @@ abstract class EntityFindBase implements EntityFind { totalUpdated++ } } - } finally { - if (eli != null) eli.close() - } + }) return totalUpdated } @@ -1495,33 +1490,27 @@ abstract class EntityFindBase implements EntityFind { } } else { this.resultSetConcurrency(ResultSet.CONCUR_UPDATABLE) - EntityListIterator eli = (EntityListIterator) null - try { - eli = iterator() + iterator().withCloseable ({eli-> + while (eli.next() != null) { // no longer need to clear cache, eli.remove() does that eli.remove() totalDeleted++ } - } finally { - if (eli != null) eli.close() - } + }) } return totalDeleted } @Override void extract(SimpleEtl etl) { - EntityListIterator eli = iterator() - try { + try (EntityListIterator eli = iterator()) { EntityValue ev while ((ev = eli.next()) != null) { etl.processEntry(ev) } } catch (StopException e) { logger.warn("EntityFind extract stopped on: " + (e.getCause()?.toString() ?: e.toString())) - } finally { - eli.close() } } diff --git a/framework/src/main/groovy/org/moqui/impl/entity/EntityListIteratorImpl.java b/framework/src/main/groovy/org/moqui/impl/entity/EntityListIteratorImpl.java index fccd72de6..f0c120292 100644 --- a/framework/src/main/groovy/org/moqui/impl/entity/EntityListIteratorImpl.java +++ b/framework/src/main/groovy/org/moqui/impl/entity/EntityListIteratorImpl.java @@ -285,6 +285,7 @@ public EntityValueBase currentEntityValueBase() { } catch (SQLException e) { throw new EntityException("Error getting all results", e); } finally { + //TODO: Remove closeAfter with respect to try-with-resource implementation if (closeAfter) close(); } } diff --git a/framework/src/main/groovy/org/moqui/impl/entity/elastic/ElasticDatasourceFactory.groovy b/framework/src/main/groovy/org/moqui/impl/entity/elastic/ElasticDatasourceFactory.groovy index 4567832c8..a532332c0 100644 --- a/framework/src/main/groovy/org/moqui/impl/entity/elastic/ElasticDatasourceFactory.groovy +++ b/framework/src/main/groovy/org/moqui/impl/entity/elastic/ElasticDatasourceFactory.groovy @@ -161,6 +161,7 @@ class ElasticDatasourceFactory implements EntityDatasourceFactory { if (checkedEntityIndexSet.contains(indexName)) return ElasticFacade.ElasticClient elasticClient = efi.ecfi.elasticFacade.getClient(clusterName) + if (elasticClient == null) throw new IllegalStateException("No ElasticClient found for cluster name " + clusterName) if (!elasticClient.indexExists(indexName)) { Map mapping = makeElasticEntityMapping(ed) // logger.warn("Creating ES Index ${indexName} with mapping: ${JsonOutput.prettyPrint(JsonOutput.toJson(mapping))}") @@ -170,7 +171,11 @@ class ElasticDatasourceFactory implements EntityDatasourceFactory { checkedEntityIndexSet.add(indexName) } - ElasticFacade.ElasticClient getElasticClient() { efi.ecfi.elasticFacade.getClient(clusterName) } + ElasticFacade.ElasticClient getElasticClient() { + ElasticFacade.ElasticClient client = efi.ecfi.elasticFacade.getClient(clusterName) + if (client == null) throw new IllegalStateException("No ElasticClient found for cluster name " + clusterName) + return client + } String getIndexName(EntityDefinition ed) { return indexPrefix + ed.getTableNameLowerCase() } diff --git a/framework/src/main/groovy/org/moqui/impl/entity/elastic/ElasticEntityListIterator.java b/framework/src/main/groovy/org/moqui/impl/entity/elastic/ElasticEntityListIterator.java index bf2c594d0..947193eaf 100644 --- a/framework/src/main/groovy/org/moqui/impl/entity/elastic/ElasticEntityListIterator.java +++ b/framework/src/main/groovy/org/moqui/impl/entity/elastic/ElasticEntityListIterator.java @@ -125,6 +125,7 @@ boolean nextResult() { // logger.warn("nextResult end resultCount " + resultCount + " overallIndex " + overallIndex + " currentListStartIndex " + currentListStartIndex + " currentDocList.size() " + currentDocList.size()); return hasCurrentValue(); } + @SuppressWarnings("unchecked") void fetchNext() { if (this.closed) throw new IllegalStateException("EntityListIterator is closed, cannot fetch next results"); diff --git a/framework/src/main/groovy/org/moqui/impl/screen/ScreenRenderImpl.groovy b/framework/src/main/groovy/org/moqui/impl/screen/ScreenRenderImpl.groovy index f3e8b8ee6..eb40bf57d 100644 --- a/framework/src/main/groovy/org/moqui/impl/screen/ScreenRenderImpl.groovy +++ b/framework/src/main/groovy/org/moqui/impl/screen/ScreenRenderImpl.groovy @@ -1105,6 +1105,19 @@ class ScreenRenderImpl implements ScreenRender { return activePath } + // TODO: This may not be the actual place we decided on, but due to lost work this is my best guess + // Get the first screen path of the parent screens with a transition specified of the currently rendered screen + String getScreenPathHasTransition(String transitionName) { + int screenPathDefListSize = screenUrlInfo.screenPathDefList.size() + for (int i = 0; i < screenPathDefListSize; i++) { + ScreenDefinition screenDef = (ScreenDefinition) screenUrlInfo.screenPathDefList.get(i) + if (screenDef.hasTransition(transitionName)) { + return '/' + screenUrlInfo.fullPathNameList.subList(0,i).join('/') + (i == 0 ? '' : '/') + } + } + return null + } + String renderSubscreen() { // first see if there is another screen def in the list if (!getActiveScreenHasNext()) { @@ -1190,7 +1203,15 @@ class ScreenRenderImpl implements ScreenRender { return "" } - String renderSectionInclude(MNode sectionIncludeNode) { + MNode getSectionIncludedNode(MNode sectionIncludeNode) { + ScreenDefinition sd = getActiveScreenDef() + String sectionName = getSectionIncludeName(sectionIncludeNode) + ScreenSection section = sd.getSection(sectionName) + if (section == null) throw new BaseArtifactException("No section with name [${sectionName}] in screen [${sd.location}]") + return section.sectionNode + } + + String getSectionIncludeName(MNode sectionIncludeNode) { String sectionLocation = sectionIncludeNode.attribute("location") String sectionName = sectionIncludeNode.attribute("name") boolean isDynamic = (sectionLocation != null && sectionLocation.contains('${')) || (sectionName != null && sectionName.contains('${')) @@ -1201,11 +1222,14 @@ class ScreenRenderImpl implements ScreenRender { String cacheName = sectionLocation + "#" + sectionName if (sd.sectionByName.get(cacheName) == null) sd.pullSectionInclude(sectionIncludeNode) // logger.warn("sd.sectionByName ${sd.sectionByName}") - return renderSection(cacheName) + return cacheName } else { - return renderSection(sectionName) + return sectionName } } + String renderSectionInclude(MNode sectionIncludeNode) { + renderSection(getSectionIncludeName(sectionIncludeNode)) + } MNode getFormNode(String formName) { FormInstance fi = getFormInstance(formName) @@ -1403,7 +1427,7 @@ class ScreenRenderImpl implements ScreenRender { String pushContext() { ec.contextStack.push(); return "" } String popContext() { ec.contextStack.pop(); return "" } - /** Call this at the beginning of a form-single or for form-list.@first-row-map and @last-row-map. Always call popContext() at the end of the form! */ + /** Call this at the beginning of a form-single or for form-list.@map-first-row and @map-last-row. Always call popContext() at the end of the form! */ String pushSingleFormMapContext(String mapExpr) { ContextStack cs = ec.contextStack Map valueMap = null @@ -1417,6 +1441,13 @@ class ScreenRenderImpl implements ScreenRender { return "" } + Map getSingleFormMap(String mapExpr) { + Map valueMap = null + if (mapExpr != null && !mapExpr.isEmpty()) valueMap = (Map) ec.resourceFacade.expression(mapExpr, null) + if (valueMap instanceof EntityValue) valueMap = ((EntityValue) valueMap).getMap() + if (valueMap == null) valueMap = new HashMap() + return valueMap + } String startFormListRow(ScreenForm.FormListRenderInfo listRenderInfo, Object listEntry, int index, boolean hasNext) { ContextStack cs = ec.contextStack cs.push() @@ -1634,7 +1665,7 @@ class ScreenRenderImpl implements ScreenRender { int afnSize = allFieldNodes.size() for (int i = 0; i < afnSize; i++) { MNode fieldNode = (MNode) allFieldNodes.get(i) - addFormFieldValue(fieldNode, fieldValues, false) + addFormFieldValue(fieldNode, fieldValues, (char) 'r') } return fieldValues } @@ -1649,7 +1680,7 @@ class ScreenRenderImpl implements ScreenRender { int afnSize = allFieldNodes.size() for (int i = 0; i < afnSize; i++) { MNode fieldNode = (MNode) allFieldNodes.get(i) - addFormFieldValue(fieldNode, fieldValues, true) + addFormFieldValue(fieldNode, fieldValues, (char) 'h') } // add orderByField @@ -1682,11 +1713,11 @@ class ScreenRenderImpl implements ScreenRender { ArrayList> outRows = new ArrayList<>(rowsSize) for (int ri = 0; ri < rowsSize; ri++) { Map row = (Map) listObject.get(ri) - outRows.add(transformFormListRow(renderInfo, row)) + outRows.add(transformFormListRow(renderInfo, row, (char) 'r')) } return outRows } - Map transformFormListRow(ScreenForm.FormListRenderInfo renderInfo, Map row) { + Map transformFormListRow(ScreenForm.FormListRenderInfo renderInfo, Map row, char rowType) { ArrayList fieldNodeList = renderInfo.getFormNode().children("field") int fieldNodeListSize = fieldNodeList.size() Set displayedFields = renderInfo.getDisplayedFields() @@ -1706,7 +1737,7 @@ class ScreenRenderImpl implements ScreenRender { cs.push(row) cs.push() try { - addFormFieldValue(fieldNode, outRow, false) + addFormFieldValue(fieldNode, outRow, rowType) } finally { cs.pop() cs.pop() @@ -1718,12 +1749,18 @@ class ScreenRenderImpl implements ScreenRender { } // NOTE: this takes a fieldValues Map as a parameter to populate because a singe form field may have multiple values - void addFormFieldValue(MNode fieldNode, Map fieldValues, boolean useHeader) { + void addFormFieldValue(MNode fieldNode, Map fieldValues, char rowType) { String fieldName = fieldNode.attribute("name") MNode activeSubNode = (MNode) null - if (useHeader) { + if (rowType == (char) 'h') { activeSubNode = fieldNode.first("header-field") + } else if (rowType == (char) 'f') { + activeSubNode = fieldNode.first("first-row-field") + } else if (rowType == (char) 's') { + activeSubNode = fieldNode.first("second-row-field") + } else if (rowType == (char) 'l') { + activeSubNode = fieldNode.first("last-row-field") } else { ArrayList condFieldNodeList = fieldNode.children("conditional-field") for (int j = 0; j < condFieldNodeList.size(); j++) { @@ -1786,6 +1823,7 @@ class ScreenRenderImpl implements ScreenRender { String fieldValue = (String) null String textAttr = widgetNode.attribute("text") String currencyAttr = widgetNode.attribute("currency-unit-field") + String currencyNoSymbolAttr = widgetNode.attribute("currency-hide-symbol") if (textAttr != null && ! textAttr.isEmpty()) { String textMapAttr = widgetNode.attribute("text-map") Map textMap = (Map) null @@ -1796,9 +1834,16 @@ class ScreenRenderImpl implements ScreenRender { } else { fieldValue = ec.resourceFacade.expand(textAttr, null) } - if (currencyAttr != null && !currencyAttr.isEmpty()) - fieldValue = ec.l10nFacade.formatCurrency(fieldValue, ec.resourceFacade.expression(currencyAttr, null) as String) + if (currencyAttr != null && !currencyAttr.isEmpty()) { + if (currencyNoSymbolAttr == "true") + fieldValue = ec.l10nFacade.formatCurrencyNoSymbol(fieldValue, ec.resourceFacade.expression(currencyAttr, null) as String) + else + fieldValue = ec.l10nFacade.formatCurrency(fieldValue, ec.resourceFacade.expression(currencyAttr, null) as String) + } } else if (currencyAttr != null && !currencyAttr.isEmpty()) { + if (currencyNoSymbolAttr == "true") + fieldValue = ec.l10nFacade.formatCurrencyNoSymbol(getFieldValue(fieldNode, ""), ec.resourceFacade.expression(currencyAttr, null) as String) + else fieldValue = ec.l10nFacade.formatCurrency(getFieldValue(fieldNode, ""), ec.resourceFacade.expression(currencyAttr, null) as String) } else { fieldValue = getFieldValueString(widgetNode) @@ -2013,7 +2058,8 @@ class ScreenRenderImpl implements ScreenRender { return transValue } - Map makeFormListSingleMap(ScreenForm.FormListRenderInfo renderInfo, Map listEntry, UrlInstance formTransitionUrl) { + Map makeFormListSingleMap(ScreenForm.FormListRenderInfo renderInfo, Map listEntry, + UrlInstance formTransitionUrl, String rowType) { MNode formNode = renderInfo.getFormNode() Map outMap = new LinkedHashMap<>() @@ -2022,7 +2068,7 @@ class ScreenRenderImpl implements ScreenRender { outMap.putAll(getFormHiddenParameters(formNode)) // listEntry fields before boilerplate fields below - Map row = transformFormListRow(renderInfo, listEntry) + Map row = transformFormListRow(renderInfo, listEntry, rowType.charAt(0)) outMap.putAll(row) outMap.put("moquiFormName", formNode.attribute("name")) @@ -2032,7 +2078,8 @@ class ScreenRenderImpl implements ScreenRender { return outMap } - Map makeFormListMultiMap(ScreenForm.FormListRenderInfo renderInfo, ArrayList> listObject, UrlInstance formTransitionUrl) { + Map makeFormListMultiMap(ScreenForm.FormListRenderInfo renderInfo, + ArrayList> listObject, UrlInstance formTransitionUrl) { MNode formNode = renderInfo.getFormNode() Map outMap = new LinkedHashMap<>() @@ -2044,7 +2091,7 @@ class ScreenRenderImpl implements ScreenRender { int listSize = listObject.size() for (int i = 0; i < listSize; i++) { Map listEntry = (Map) listObject.get(i) - Map row = transformFormListRow(renderInfo, listEntry) + Map row = transformFormListRow(renderInfo, listEntry, (char) 'r') for (Map.Entry mapEntry in row.entrySet()) { outMap.put(mapEntry.getKey() + "_" + i, mapEntry.getValue()) } @@ -2323,6 +2370,7 @@ class ScreenRenderImpl implements ScreenRender { boolean active = (nextItem == subscreensItem.name) Map itemMap = [name:subscreensItem.name, title:ec.resource.expand(subscreensItem.menuTitle, ""), path:screenPath, pathWithParams:pathWithParams, image:image, imageType:imageType] + if (subscreensItem.menuInclude) itemMap.menuInclude = true if (active) itemMap.active = true if (screenUrlInstance.disableLink) itemMap.disableLink = true subscreensList.add(itemMap) diff --git a/framework/src/main/groovy/org/moqui/impl/screen/ScreenUrlInfo.groovy b/framework/src/main/groovy/org/moqui/impl/screen/ScreenUrlInfo.groovy index 3ce33c96e..ec31ef405 100644 --- a/framework/src/main/groovy/org/moqui/impl/screen/ScreenUrlInfo.groovy +++ b/framework/src/main/groovy/org/moqui/impl/screen/ScreenUrlInfo.groovy @@ -271,6 +271,9 @@ class ScreenUrlInfo { ArrayDeque artifactExecutionInfoStack = new ArrayDeque() int screenPathDefListSize = screenPathDefList.size() + boolean allowedByScreenDefinitionView = false + boolean allowedByScreenDefinitionAll = false + boolean allowedByScreenDefinition = false for (int i = 0; i < screenPathDefListSize; i++) { AuthzAction curActionEnum = (i == (screenPathDefListSize - 1)) ? actionEnum : ArtifactExecutionInfo.AUTHZA_VIEW ScreenDefinition screenDef = (ScreenDefinition) screenPathDefList.get(i) @@ -285,9 +288,15 @@ class ScreenUrlInfo { MNode screenNode = screenDef.getScreenNode() String requireAuthentication = screenNode.attribute('require-authentication') + allowedByScreenDefinitionView = "anonymous-view".equals(requireAuthentication) + allowedByScreenDefinitionAll = "anonymous-all".equals(requireAuthentication) + if (actionEnum == ArtifactExecutionInfo.AUTHZA_VIEW) { + allowedByScreenDefinition = allowedByScreenDefinition || allowedByScreenDefinitionView || allowedByScreenDefinitionAll + } else if (actionEnum == ArtifactExecutionInfo.AUTHZA_ALL) + allowedByScreenDefinition = allowedByScreenDefinition || allowedByScreenDefinitionAll if (!aefi.isPermitted(aeii, lastAeii, isLast ? (!requireAuthentication || "true".equals(requireAuthentication)) : false, false, false, artifactExecutionInfoStack)) { - // logger.warn("TOREMOVE user ${username} is NOT allowed to view screen at path ${this.fullPathNameList} because of screen at ${screenDef.location}") + //logger.warn("TOREMOVE user ${userId} is NOT allowed to view screen at path ${this.fullPathNameList} because of screen at ${screenDef.location}") if (permittedCacheKey != null) aefi.screenPermittedCache.put(permittedCacheKey, false) return false } @@ -296,7 +305,7 @@ class ScreenUrlInfo { } // see if the transition is permitted - if (transitionItem != null) { + if (!allowedByScreenDefinition && transitionItem != null) { ScreenDefinition lastScreenDef = (ScreenDefinition) screenPathDefList.get(screenPathDefList.size() - 1) ArtifactExecutionInfoImpl aeii = new ArtifactExecutionInfoImpl("${lastScreenDef.location}/${transitionItem.name}", ArtifactExecutionInfo.AT_XML_SCREEN_TRANS, ArtifactExecutionInfo.AUTHZA_VIEW, null) @@ -317,10 +326,16 @@ class ScreenUrlInfo { if (authzAction == null) authzAction = ServiceDefinition.verbAuthzActionEnumMap.get(ServiceDefinition.getVerbFromName(serviceName)) if (authzAction == null) authzAction = ArtifactExecutionInfo.AUTHZA_ALL + boolean allowedByServiceDefinition = false + if (authzAction == ArtifactExecutionInfo.AUTHZA_VIEW) { + allowedByServiceDefinition = allowedByScreenDefinitionView || (sd != null && ("anonymous-view".equals(sd.authenticate) || "anonymous-all".equals(sd.authenticate))) + } else if (authzAction in [ArtifactExecutionInfo.AUTHZA_ALL, ArtifactExecutionInfo.AUTHZA_CREATE, ArtifactExecutionInfo.AUTHZA_UPDATE, ArtifactExecutionInfo.AUTHZA_DELETE]) { + allowedByServiceDefinition = allowedByScreenDefinitionAll || (sd != null && "anonymous-all".equals(sd.authenticate)) + } ArtifactExecutionInfoImpl aeii = new ArtifactExecutionInfoImpl(serviceName, ArtifactExecutionInfo.AT_SERVICE, authzAction, null) ArtifactExecutionInfoImpl lastAeii = (ArtifactExecutionInfoImpl) artifactExecutionInfoStack.peekFirst() - if (!aefi.isPermitted(aeii, lastAeii, true, false, false, null)) { + if (!aefi.isPermitted(aeii, lastAeii, !allowedByServiceDefinition, false, false, null)) { // logger.warn("TOREMOVE user ${username} is NOT allowed to run transition at path ${this.fullPathNameList} because of screen at ${screenDef.location}") if (permittedCacheKey != null) aefi.screenPermittedCache.put(permittedCacheKey, false) return false diff --git a/framework/src/main/groovy/org/moqui/impl/service/ServiceCallAsyncImpl.groovy b/framework/src/main/groovy/org/moqui/impl/service/ServiceCallAsyncImpl.groovy index edf7903bb..3d4ddceeb 100644 --- a/framework/src/main/groovy/org/moqui/impl/service/ServiceCallAsyncImpl.groovy +++ b/framework/src/main/groovy/org/moqui/impl/service/ServiceCallAsyncImpl.groovy @@ -102,6 +102,12 @@ class ServiceCallAsyncImpl extends ServiceCallImpl implements ServiceCallAsync { this.serviceName = serviceName this.parameters = new HashMap<>(parameters) } + AsyncServiceInfo(ExecutionContextFactoryImpl ecfi, String username, String serviceName, Map parameters) { + ecfiLocal = ecfi + threadUsername = username + this.serviceName = serviceName + this.parameters = new HashMap<>(parameters) + } @Override void writeExternal(ObjectOutput out) throws IOException { @@ -123,6 +129,9 @@ class ServiceCallAsyncImpl extends ServiceCallImpl implements ServiceCallAsync { } Map runInternal() throws Exception { + return runInternal(null, false) + } + Map runInternal(Map parameters, boolean skipEcCheck) throws Exception { ExecutionContextImpl threadEci = (ExecutionContextImpl) null try { // check for active Transaction @@ -135,22 +144,33 @@ class ServiceCallAsyncImpl extends ServiceCallImpl implements ServiceCallAsync { } } // check for active ExecutionContext - ExecutionContextImpl activeEc = getEcfi().activeContext.get() - if (activeEc != null) { - logger.error("In ServiceCallAsync service ${serviceName} there is already an ExecutionContext for user ${activeEc.user.username} (from ${activeEc.forThreadId}:${activeEc.forThreadName}) in this thread ${Thread.currentThread().id}:${Thread.currentThread().name}, destroying") - try { - activeEc.destroy() - } catch (Throwable t) { - logger.error("Error destroying ExecutionContext already in place in ServiceCallAsync in thread ${Thread.currentThread().id}:${Thread.currentThread().name}", t) + if (!skipEcCheck) { + ExecutionContextImpl activeEc = getEcfi().activeContext.get() + if (activeEc != null) { + logger.error("In ServiceCallAsync service ${serviceName} there is already an ExecutionContext for user ${activeEc.user.username} (from ${activeEc.forThreadId}:${activeEc.forThreadName}) in this thread ${Thread.currentThread().id}:${Thread.currentThread().name}, destroying") + try { + activeEc.destroy() + } catch (Throwable t) { + logger.error("Error destroying ExecutionContext already in place in ServiceCallAsync in thread ${Thread.currentThread().id}:${Thread.currentThread().name}", t) + } } } threadEci = getEcfi().getEci() - if (threadUsername != null && threadUsername.length() > 0) + if (threadUsername != null && threadUsername.length() > 0) { threadEci.userFacade.internalLoginUser(threadUsername, false) + } else { + threadEci.userFacade.loginAnonymousIfNoUser() + } + + Map parmsToUse = this.parameters + if (parameters != null) { + parmsToUse = new HashMap<>(this.parameters) + parmsToUse.putAll(parameters) + } // NOTE: authz is disabled because authz is checked before queueing - Map result = threadEci.serviceFacade.sync().name(serviceName).parameters(parameters).disableAuthz().call() + Map result = threadEci.serviceFacade.sync().name(serviceName).parameters(parmsToUse).disableAuthz().call() return result } catch (Throwable t) { logger.error("Error in async service", t) diff --git a/framework/src/main/groovy/org/moqui/impl/service/ServiceFacadeImpl.groovy b/framework/src/main/groovy/org/moqui/impl/service/ServiceFacadeImpl.groovy index f28eca2cc..da7bf9357 100644 --- a/framework/src/main/groovy/org/moqui/impl/service/ServiceFacadeImpl.groovy +++ b/framework/src/main/groovy/org/moqui/impl/service/ServiceFacadeImpl.groovy @@ -14,7 +14,11 @@ package org.moqui.impl.service import groovy.transform.CompileStatic +import org.moqui.Moqui +import org.moqui.impl.context.ArtifactExecutionInfoImpl +import org.moqui.impl.context.ArtifactExecutionInfoImpl.ArtifactTypeStats import org.moqui.impl.context.ContextJavaUtil +import org.moqui.impl.context.ContextJavaUtil.CustomScheduledExecutor import org.moqui.resource.ResourceReference import org.moqui.context.ToolFactory import org.moqui.impl.context.ExecutionContextFactoryImpl @@ -25,6 +29,7 @@ import org.moqui.impl.service.runner.RemoteJsonRpcServiceRunner import org.moqui.service.* import org.moqui.util.CollectionUtilities import org.moqui.util.MNode +import org.moqui.util.ObjectUtilities import org.moqui.util.RestClient import org.moqui.util.StringUtilities import org.slf4j.Logger @@ -32,7 +37,9 @@ import org.slf4j.LoggerFactory import javax.cache.Cache import javax.mail.internet.MimeMessage +import java.sql.Timestamp import java.util.concurrent.* +import java.util.concurrent.atomic.AtomicInteger import java.util.concurrent.locks.ReentrantLock @CompileStatic @@ -52,6 +59,7 @@ class ServiceFacadeImpl implements ServiceFacade { private ScheduledJobRunner jobRunner = null public final ThreadPoolExecutor jobWorkerPool + private LoadRunner loadRunner = null /** Distributed ExecutorService for async services, etc */ protected ExecutorService distributedExecutorService = null @@ -599,4 +607,353 @@ class ServiceFacadeImpl implements ServiceFacade { if (callbackList != null && callbackList.size() > 0) for (ServiceCallback scb in callbackList) scb.receiveEvent(context, t) } + + // ========================== + // Service LoadRunner Classes + // ========================== + + synchronized LoadRunner getLoadRunner() { + if (loadRunner == null) loadRunner = new LoadRunner(ecfi) + return loadRunner + } + + static class LoadRunnerServiceRunnable implements Runnable, Externalizable { + volatile ExecutionContextFactoryImpl ecfi + volatile LoadRunner loadRunner + String serviceName, parametersExpr + + LoadRunnerServiceRunnable() { + // init the other objects that can't be serialized + ecfi = (ExecutionContextFactoryImpl) Moqui.getExecutionContextFactory() + loadRunner = ecfi.serviceFacade.getLoadRunner() + } + LoadRunnerServiceRunnable(String serviceName, String parametersExpr, LoadRunner loadRunner) { + this.loadRunner = loadRunner + this.ecfi = loadRunner.ecfi + this.serviceName = serviceName + this.parametersExpr = parametersExpr + } + + @Override + void writeExternal(ObjectOutput out) throws IOException { + out.writeUTF(serviceName) // never null + out.writeObject(parametersExpr) // may be null + } + + @Override + void readExternal(ObjectInput objectInput) throws IOException, ClassNotFoundException { + serviceName = objectInput.readUTF() + parametersExpr = objectInput.readObject() + } + + @Override void run() { + try { + runInternal() + } catch (Throwable t) { + logger.error("Error in LoadRunner service run", t) + } + } + void runInternal() { + // check for active Transaction + if (getEcfi().transactionFacade.isTransactionInPlace()) { + logger.error("In LoadRunner service ${serviceName} a transaction is in place for thread ${Thread.currentThread().getName()}, trying to commit") + try { + getEcfi().transactionFacade.destroyAllInThread() + } catch (Exception e) { + logger.error("LoadRunner commit in place transaction failed for thread ${Thread.currentThread().getName()}", e) + } + } + // check for active ExecutionContext + ExecutionContextImpl activeEc = getEcfi().activeContext.get() + if (activeEc != null) { + logger.error("In LoadRunner service ${serviceName} there is already an ExecutionContext for user ${activeEc.user.username} (from ${activeEc.forThreadId}:${activeEc.forThreadName}) in this thread ${Thread.currentThread().id}:${Thread.currentThread().name}, destroying") + try { + activeEc.destroy() + } catch (Throwable t) { + logger.error("Error destroying LoadRunner already in place in ServiceCallAsync in thread ${Thread.currentThread().id}:${Thread.currentThread().name}", t) + } + } + + LoadRunnerServiceInfo serviceInfo = loadRunner.getServiceInfo(serviceName, parametersExpr) + if (serviceInfo == null) { + logger.error("Service Info not found for ${serviceName} ${parametersExpr}, not running") + return + } + String parametersExpr = serviceInfo.parametersExpr + Map parameters = [index:loadRunner.execIndex.getAndIncrement()] as Map + if (parametersExpr != null && !parametersExpr.isEmpty()) { + try { + Map exprMap = (Map) loadRunner.ecfi.getEci().resourceFacade + .expression(parametersExpr, null, parameters) + if (exprMap != null) parameters.putAll(exprMap) + } catch (Throwable t) { + logger.error("Error in Service LoadRunner parameter expression: ${parametersExpr}", t) + } + } + + // before starting, and tracking the startTime, do a small random delay for variation in run times + if (serviceInfo.runDelayVaryMs != 0) + Thread.sleep(ThreadLocalRandom.current().nextInt(serviceInfo.runDelayVaryMs)) + + long startTime = System.currentTimeMillis() + ExecutionContextImpl threadEci = ecfi.getEci() + try { + // always login anonymous, disable authz below + threadEci.userFacade.loginAnonymousIfNoUser() + + // run the service + try { + serviceInfo.lastResult = threadEci.serviceFacade.sync().name(serviceName) + .parameters(parameters).disableAuthz().call() + } catch (Throwable t) { + // logged elsewhere, just count and swallow + serviceInfo.errorCount++ + } + + // count the run and accumulate stats + serviceInfo.countRun(loadRunner, startTime, System.currentTimeMillis(), + threadEci.artifactExecutionFacade.getArtifactTypeStats()) + } finally { + if (threadEci != null) threadEci.destroy() + } + } + } + static class LoadRunnerServiceStats { + long lastRunTime = 0, beginTime = 0, totalTime = 0, totalSquaredTime = 0, minTime = Long.MAX_VALUE, maxTime = 0 + int runCount = 0, errorCount = 0 + ArtifactTypeStats artifactTypeStats = new ArtifactTypeStats() + Map getMap() { + Map newMap = [lastRunTime:lastRunTime, beginTime:beginTime, totalTime:totalTime, + totalSquaredTime:totalSquaredTime, minTime:minTime, maxTime:maxTime, runCount:runCount, errorCount:errorCount] as Map + newMap.put("artifactTypeStats", ObjectUtilities.objectToMap(artifactTypeStats)) + return newMap + } + } + static class LoadRunnerServiceInfo extends LoadRunnerServiceStats { + String serviceName, parametersExpr + int targetThreads, runDelayMs, runDelayVaryMs, rampDelayMs, timeBinLength, timeBinsKeep + AtomicInteger currentThreads = new AtomicInteger(0) + + Map lastResult = null + ConcurrentLinkedDeque timeBinList = new ConcurrentLinkedDeque<>() + ArrayList runFutures = new ArrayList<>() + ScheduledFuture rampFuture = null + + LoadRunnerServiceInfo(String serviceName, String parametersExpr, int targetThreads, + int runDelayMs, int runDelayVaryMs, int rampDelayMs, int timeBinLength, int timeBinsKeep) { + this.serviceName = serviceName; this.parametersExpr = parametersExpr + this.targetThreads = targetThreads + this.runDelayMs = runDelayMs; this.runDelayVaryMs = runDelayVaryMs; this.rampDelayMs = rampDelayMs + this.timeBinLength = timeBinLength; this.timeBinsKeep = timeBinsKeep + } + + void countRun(LoadRunner loadRunner, long startTime, long endTime, ArtifactTypeStats stats) { + long runTime = endTime - startTime + // logger.info("count run ${serviceName} ${runTime} ${Thread.currentThread().name}") + LoadRunnerServiceStats curBin = null + + // find the current time bin in a semaphore locked section, the rest is increments and can be run multithreaded + loadRunner.mutateLock.lock() + // logger.info("count run after lock ${serviceName} ${runTime} ${Thread.currentThread().name}") + try { + if (beginTime == 0) beginTime = startTime + + curBin = timeBinList.isEmpty() ? null : timeBinList.getLast() + // create and add a new bin if there are none or if this hit is after the bin's end time (need to advance the bin) + if (curBin == null || curBin.beginTime + timeBinLength < startTime) { + curBin = new LoadRunnerServiceStats() + curBin.beginTime = startTime + timeBinList.add(curBin) + if (timeBinList.size() > timeBinsKeep) { + LoadRunnerServiceStats removeBin = timeBinList.removeFirst() + // logger.info("Removed time bin starting ${new Timestamp(removeBin.beginTime)} count ${removeBin.runCount}") + } + } + + // some exceptions, these are multiple operations and need to be in the locked section to be accurate, maybe don't need to be... + if (runTime < this.minTime) this.minTime = runTime + if (runTime > this.maxTime) this.maxTime = runTime + if (runTime < curBin.minTime) curBin.minTime = runTime + if (runTime > curBin.maxTime) curBin.maxTime = runTime + } finally { + loadRunner.mutateLock.unlock() + // logger.info("count run after unlock ${serviceName} ${runTime} ${Thread.currentThread().name}") + // loadRunner.logFutures() + } + + // for all runs + this.runCount++ + this.lastRunTime = endTime + this.totalTime += runTime + this.totalSquaredTime += runTime * runTime + this.artifactTypeStats.add(stats) + + // same thing for just this bin + curBin.runCount++ + curBin.lastRunTime = endTime + curBin.totalTime += runTime + curBin.totalSquaredTime += runTime * runTime + curBin.artifactTypeStats.add(stats) + } + + void addThread(LoadRunner loadRunner) { + LoadRunnerServiceRunnable runnable = new LoadRunnerServiceRunnable(serviceName, parametersExpr, loadRunner) + // NOTE: use scheduleWithFixedDelay so delay is wait between terminate of one and start of another, better for both short and long running test runs + ScheduledFuture future = loadRunner.scheduledExecutor.scheduleWithFixedDelay(runnable, 1, runDelayMs, TimeUnit.MILLISECONDS) + // logger.info("Added run thread runDelayMs ${runDelayMs} ${runDelayMs?.class?.name} done ${future.done} ${future.toString()}") + runFutures.add(future) + } + void addRampThread(LoadRunner loadRunner) { + if (rampFuture != null && !rampFuture) + beginTime = System.currentTimeMillis() + LoadRunnerRamperRunnable runnable = new LoadRunnerRamperRunnable(loadRunner, this) + // NOTE: use scheduleAtFixedRate so one is added each delay period regardless of how long it takes (generally not long) + rampFuture = loadRunner.scheduledExecutor.scheduleAtFixedRate(runnable, 1, rampDelayMs, TimeUnit.MILLISECONDS) + } + void resetStats() { + lastRunTime = 0; beginTime = 0; totalTime = 0; totalSquaredTime = 0; minTime = Long.MAX_VALUE; maxTime = 0 + runCount = 0; errorCount = 0 + artifactTypeStats = new ArtifactTypeStats() + lastResult = null + timeBinList = new ConcurrentLinkedDeque<>() + } + } + static class LoadRunnerRamperRunnable implements Runnable { + LoadRunner loadRunner + LoadRunnerServiceInfo serviceInfo + LoadRunnerRamperRunnable(LoadRunner loadRunner, LoadRunnerServiceInfo serviceInfo) { + this.loadRunner = loadRunner + this.serviceInfo = serviceInfo + } + @Override void run() { + if (serviceInfo.currentThreads < serviceInfo.targetThreads) { + serviceInfo.addThread(loadRunner) + // may not actually need AtomicInteger here, but for ramp down will need a list of ScheduledFuture objects and might be useful there + serviceInfo.currentThreads.incrementAndGet() + } + // TODO add delayed ramp-down, useful for some performance behavior patterns but usually redundant with delayed ramp up to look for elbows in the response time over time + } + } + static class LoadRunnerThreadFactory implements ThreadFactory { + private final ThreadGroup workerGroup = new ThreadGroup("LoadRunner") + private final AtomicInteger threadNumber = new AtomicInteger(1) + Thread newThread(Runnable r) { return new Thread(workerGroup, r, "LoadRunner-" + threadNumber.getAndIncrement()) } + } + + static class LoadRunner { + ExecutionContextFactoryImpl ecfi + CustomScheduledExecutor scheduledExecutor = null + ArrayList serviceInfos = new ArrayList<>() + Integer corePoolSize = 4, maxPoolSize = null + AtomicInteger execIndex = new AtomicInteger(1) + ReentrantLock mutateLock = new ReentrantLock() + + LoadRunner(ExecutionContextFactoryImpl ecfi) { + this.ecfi = ecfi + } + + LoadRunnerServiceInfo getServiceInfo(String serviceName, String parametersExpr) { + for (int i = 0; i < serviceInfos.size(); i++) { + LoadRunnerServiceInfo curInfo = (LoadRunnerServiceInfo) serviceInfos.get(i) + if (curInfo.serviceName == serviceName && curInfo.parametersExpr == parametersExpr) + return curInfo + } + return null + } + void setServiceInfo(String serviceName, String parametersExpr, int targetThreads, int runDelayMs, + int runDelayVaryMs, int rampDelayMs, int timeBinLength, int timeBinsKeep) { + mutateLock.lock() + try { + LoadRunnerServiceInfo serviceInfo = getServiceInfo(serviceName, parametersExpr) + if (serviceInfo == null) { + serviceInfo = new LoadRunnerServiceInfo(serviceName, parametersExpr, targetThreads, + runDelayMs, runDelayVaryMs, rampDelayMs, timeBinLength, timeBinsKeep) + + serviceInfos.add(serviceInfo) + + if (scheduledExecutor != null) { + // begin() already called, get this started + serviceInfo.addRampThread(this) + } + } else { + serviceInfo.targetThreads = targetThreads + serviceInfo.runDelayMs = runDelayMs + serviceInfo.rampDelayMs = rampDelayMs + serviceInfo.timeBinLength = timeBinLength + serviceInfo.timeBinsKeep = timeBinsKeep + } + } finally { + mutateLock.unlock() + } + } + void begin() { + mutateLock.lock() + try { + if (scheduledExecutor == null) { + // restart index + execIndex = new AtomicInteger(1) + + for (int i = 0; i < serviceInfos.size(); i++) { + LoadRunnerServiceInfo curInfo = (LoadRunnerServiceInfo) serviceInfos.get(i) + // clear out stats before start + curInfo.currentThreads = new AtomicInteger(0) + curInfo.resetStats() + } + + scheduledExecutor = new CustomScheduledExecutor(corePoolSize, new LoadRunnerThreadFactory()) + if (maxPoolSize == null) maxPoolSize = Runtime.getRuntime().availableProcessors() * 4 + scheduledExecutor.setMaximumPoolSize(maxPoolSize) + + for (int i = 0; i < serviceInfos.size(); i++) { + LoadRunnerServiceInfo curInfo = (LoadRunnerServiceInfo) serviceInfos.get(i) + // get the ramp thread started + curInfo.addRampThread(this) + } + } + } finally { + mutateLock.unlock() + } + } + void stopNow() { + mutateLock.lock() + try { + if (scheduledExecutor != null) { + logger.info("Shutting down LoadRunner ScheduledExecutorService now") + scheduledExecutor.shutdownNow() + scheduledExecutor = null + } + } finally { + mutateLock.unlock() + } + } + void stopWait() { + mutateLock.lock() + try { + if (scheduledExecutor != null) { + logger.info("Shutting down LoadRunner ScheduledExecutorService") + scheduledExecutor.shutdown() + } + + if (scheduledExecutor != null) { + scheduledExecutor.awaitTermination(30, TimeUnit.SECONDS) + if (scheduledExecutor.isTerminated()) logger.info("LoadRunner Scheduled executor shut down and terminated") + else logger.warn("LoadRunner Scheduled executor NOT YET terminated, waited 30 seconds") + scheduledExecutor = null + } + } finally { + mutateLock.unlock() + } + } + + void logFutures() { + for (int si = 0; si < serviceInfos.size(); si++) { + LoadRunnerServiceInfo serviceInfo = serviceInfos.get(si) + logger.info("LoadRunner RAMP Future done ${serviceInfo.rampFuture?.done} canceled ${serviceInfo.rampFuture?.cancelled} ${serviceInfo.rampFuture?.toString()}") + for (int i = 0; i < serviceInfo.runFutures.size(); i++) { + ScheduledFuture future = serviceInfo.runFutures.get(i) + logger.info("LoadRunner RUN Future done ${future.done} canceled ${future.cancelled} ${future.toString()}") + } + } + } + } } diff --git a/framework/src/main/groovy/org/moqui/impl/service/runner/EntityAutoServiceRunner.groovy b/framework/src/main/groovy/org/moqui/impl/service/runner/EntityAutoServiceRunner.groovy index 58b5fbe51..e84b4c405 100644 --- a/framework/src/main/groovy/org/moqui/impl/service/runner/EntityAutoServiceRunner.groovy +++ b/framework/src/main/groovy/org/moqui/impl/service/runner/EntityAutoServiceRunner.groovy @@ -533,7 +533,6 @@ class EntityAutoServiceRunner implements ServiceRunner { if ("*".equals(newParms.get(fieldName))) { hasWildcard = true newParms.remove(fieldName) - break } } if (hasWildcard) { diff --git a/framework/src/main/java/org/moqui/context/ExecutionContextFactory.java b/framework/src/main/java/org/moqui/context/ExecutionContextFactory.java index 38f713d0f..901a5e30b 100644 --- a/framework/src/main/java/org/moqui/context/ExecutionContextFactory.java +++ b/framework/src/main/java/org/moqui/context/ExecutionContextFactory.java @@ -34,7 +34,9 @@ public interface ExecutionContextFactory { /** Destroy the active Execution Context. When another is requested in this thread a new one will be created. */ void destroyActiveExecutionContext(); - /** Called after construction but before registration with Moqui/Servlet, check for empty database and load configured data. */ + /** Called after construction but before registration with Moqui/Servlet, check for empty database and load configured data. + * If empty-db-load is not done and on-start-load-types has a value handles that as well. + * Also loads type 'test' data if instance_purpose=test. */ boolean checkEmptyDb(); /** Destroy this ExecutionContextFactory and all resources it uses (all facades, tools, etc) */ void destroy(); diff --git a/framework/src/main/java/org/moqui/context/L10nFacade.java b/framework/src/main/java/org/moqui/context/L10nFacade.java index 235a9cf14..f2597e5d0 100644 --- a/framework/src/main/java/org/moqui/context/L10nFacade.java +++ b/framework/src/main/java/org/moqui/context/L10nFacade.java @@ -41,11 +41,16 @@ public interface L10nFacade { * @param uomId The uomId (ISO currency code), required. * @param fractionDigits Number of digits after the decimal point to display. If null defaults to number defined * by java.util.Currency.defaultFractionDigits() for the specified currency in uomId. + * @param locale Locale to use for formatting. + * @param hideSymbol option to hide the Symbol of the currency and only display the number formatted according + * to locale. * @return The formatted currency amount. */ + String formatCurrency(Object amount, String uomId, Integer fractionDigits, Locale locale, boolean hideSymbol); + String formatCurrency(Object amount, String uomId, Integer fractionDigits, Locale locale); String formatCurrency(Object amount, String uomId, Integer fractionDigits); String formatCurrency(Object amount, String uomId); - String formatCurrency(Object amount, String uomId, Integer fractionDigits, Locale locale); + String formatCurrencyNoSymbol(Object amount, String uomId); /** Round currency according to the currency's specified amount of digits and rounding method. * @param amount The amount in BigDecimal to be rounded. diff --git a/framework/src/main/java/org/moqui/context/NotificationMessage.java b/framework/src/main/java/org/moqui/context/NotificationMessage.java index b478e6121..38cb9636e 100644 --- a/framework/src/main/java/org/moqui/context/NotificationMessage.java +++ b/framework/src/main/java/org/moqui/context/NotificationMessage.java @@ -38,6 +38,9 @@ enum NotificationType { info, success, warning, danger } NotificationMessage topic(String topic); String getTopic(); + NotificationMessage subTopic(String subTopic); + String getSubTopic(); + /** Set the message as a JSON String. The top-level should be a Map (JSON Object). * @param messageJson The message as a JSON string containing a Map (JSON Object) * @return Self-reference for convenience diff --git a/framework/src/main/java/org/moqui/entity/EntityCondition.java b/framework/src/main/java/org/moqui/entity/EntityCondition.java index e3379e182..ef936ba87 100644 --- a/framework/src/main/java/org/moqui/entity/EntityCondition.java +++ b/framework/src/main/java/org/moqui/entity/EntityCondition.java @@ -37,6 +37,9 @@ public interface EntityCondition extends Externalizable { ComparisonOperator NOT_BETWEEN = ComparisonOperator.NOT_BETWEEN; ComparisonOperator LIKE = ComparisonOperator.LIKE; ComparisonOperator NOT_LIKE = ComparisonOperator.NOT_LIKE; + ComparisonOperator BEGINS_FIELD = ComparisonOperator.BEGINS_FIELD; + ComparisonOperator ENDS_FIELD = ComparisonOperator.ENDS_FIELD; + ComparisonOperator CONTAINS_FIELD = ComparisonOperator.CONTAINS_FIELD; ComparisonOperator IS_NULL = ComparisonOperator.IS_NULL; ComparisonOperator IS_NOT_NULL = ComparisonOperator.IS_NOT_NULL; @@ -45,6 +48,7 @@ public interface EntityCondition extends Externalizable { enum ComparisonOperator { EQUALS, NOT_EQUAL, LESS_THAN, GREATER_THAN, LESS_THAN_EQUAL_TO, GREATER_THAN_EQUAL_TO, + BEGINS_FIELD, ENDS_FIELD, CONTAINS_FIELD, IN, NOT_IN, BETWEEN, NOT_BETWEEN, LIKE, NOT_LIKE, IS_NULL, IS_NOT_NULL } enum JoinOperator { AND, OR } diff --git a/framework/src/main/java/org/moqui/entity/EntityConditionFactory.java b/framework/src/main/java/org/moqui/entity/EntityConditionFactory.java index bae6dd04a..5e66a2248 100644 --- a/framework/src/main/java/org/moqui/entity/EntityConditionFactory.java +++ b/framework/src/main/java/org/moqui/entity/EntityConditionFactory.java @@ -34,6 +34,7 @@ public interface EntityConditionFactory { EntityCondition makeCondition(String fieldName, EntityCondition.ComparisonOperator operator, Object value, boolean orNull); EntityCondition makeConditionToField(String fieldName, EntityCondition.ComparisonOperator operator, String toFieldName); + EntityCondition makeConditionToField(String fieldName, EntityCondition.ComparisonOperator operator, String toFieldName, Object value); /** Default to JoinOperator of AND */ EntityCondition makeCondition(List conditionList); diff --git a/framework/src/main/java/org/moqui/entity/EntityListIterator.java b/framework/src/main/java/org/moqui/entity/EntityListIterator.java index 639431001..7dea9643a 100644 --- a/framework/src/main/java/org/moqui/entity/EntityListIterator.java +++ b/framework/src/main/java/org/moqui/entity/EntityListIterator.java @@ -20,7 +20,7 @@ * Entity Cursor List Iterator for Handling Cursored Database Results */ @SuppressWarnings("unused") -public interface EntityListIterator extends ListIterator { +public interface EntityListIterator extends ListIterator, AutoCloseable { /** Close the underlying ResultSet and Connection. This must ALWAYS be called when done with an EntityListIterator. */ void close() throws EntityException; diff --git a/framework/src/main/java/org/moqui/util/CollectionUtilities.java b/framework/src/main/java/org/moqui/util/CollectionUtilities.java index 3dfae7570..3252956a8 100644 --- a/framework/src/main/java/org/moqui/util/CollectionUtilities.java +++ b/framework/src/main/java/org/moqui/util/CollectionUtilities.java @@ -593,6 +593,9 @@ public static void paginateList(String listName, String pageListName, Map context) { + // if this exists then was already paginated so don't do a subList() + if (context.containsKey(pageListName + "AlreadyPaginated")) return theList; + Integer pageRangeLow = (Integer) context.get(pageListName + "PageRangeLow"); Integer pageRangeHigh = (Integer) context.get(pageListName + "PageRangeHigh"); if (pageRangeLow == null || pageRangeHigh == null) { @@ -619,20 +622,25 @@ public static Map paginateParameters(int listSize, String pageListName, Map count) pageRangeLow = count + 1; - int pageRangeHigh = (pageIndex * pageSize) + pageSize; - if (pageRangeHigh > count) pageRangeHigh = count; - - context.put(pageListName + "Count", count); - context.put(pageListName + "PageIndex", pageIndex); - context.put(pageListName + "PageSize", pageSize); - context.put(pageListName + "PageMaxIndex", maxIndex); - context.put(pageListName + "PageRangeLow", pageRangeLow); - context.put(pageListName + "PageRangeHigh", pageRangeHigh); + // NOTE: if context has a *Count field don't calc and set values, assume are already in place + if (context.get(pageListName + "Count") == null) { + int count = listSize; + // calculate the pagination values + int maxIndex = (new BigDecimal(count - 1)).divide(new BigDecimal(pageSize), 0, RoundingMode.DOWN).intValue(); + int pageRangeLow = (pageIndex * pageSize) + 1; + if (pageRangeLow > count) pageRangeLow = count + 1; + int pageRangeHigh = (pageIndex * pageSize) + pageSize; + if (pageRangeHigh > count) pageRangeHigh = count; + + context.put(pageListName + "Count", count); + context.put(pageListName + "PageIndex", pageIndex); + context.put(pageListName + "PageSize", pageSize); + context.put(pageListName + "PageMaxIndex", maxIndex); + context.put(pageListName + "PageRangeLow", pageRangeLow); + context.put(pageListName + "PageRangeHigh", pageRangeHigh); + } else { + context.put(pageListName + "AlreadyPaginated", true); + } return context; } diff --git a/framework/src/main/java/org/moqui/util/ObjectUtilities.java b/framework/src/main/java/org/moqui/util/ObjectUtilities.java index 1be291e92..16a9c2754 100644 --- a/framework/src/main/java/org/moqui/util/ObjectUtilities.java +++ b/framework/src/main/java/org/moqui/util/ObjectUtilities.java @@ -18,7 +18,14 @@ import org.slf4j.Logger; import org.slf4j.LoggerFactory; +import java.beans.BeanInfo; +import java.beans.IntrospectionException; +import java.beans.Introspector; +import java.beans.PropertyDescriptor; import java.io.*; +import java.lang.reflect.Field; +import java.lang.reflect.InvocationTargetException; +import java.lang.reflect.Method; import java.math.BigDecimal; import java.nio.charset.Charset; import java.nio.charset.StandardCharsets; @@ -53,6 +60,47 @@ public class ObjectUtilities { temporalUnitByUomId = tum; } + /** Populate a Map with public fields and Java Bean style properties (using java.beans.BeanInfo) */ + public static Map objectToMap(Object bean) { + if (bean == null) return null; + Map map = new HashMap<>(); + + Class clazz = bean.getClass(); + Field[] fields = clazz.getFields(); + for (int fi = 0; fi < fields.length; fi++) { + Field field = fields[fi]; + try { + map.put(field.getName(), field.get(bean)); + } catch (IllegalAccessException e) { + // do nothing, maybe log at some point if we care enough and are okay with the potential performance hit + } + } + + /* commenting for now, seems to call a bunch of undesired methods, will need some work to filter them out: + try { + BeanInfo beanInfo = Introspector.getBeanInfo(clazz); + PropertyDescriptor[] propertyDescriptors = beanInfo.getPropertyDescriptors(); + for (int pi = 0; pi < propertyDescriptors.length; pi++) { + PropertyDescriptor propertyDescriptor = propertyDescriptors[pi]; + Method readMethod = propertyDescriptor.getReadMethod(); + if (readMethod != null) { + try { + map.put(propertyDescriptor.getName(), readMethod.invoke(bean)); + } catch (IllegalAccessException | InvocationTargetException e) { + // nothing again + } + } + } + } catch (IntrospectionException e) { + // nothing again + } + */ + + // this gets picked up automatically, just remove at the end, faster than checking along the way + map.remove("class"); + + return map; + } @SuppressWarnings("unchecked") public static Object basicConvert(Object value, final String javaType) { diff --git a/framework/src/main/resources/MoquiDefaultConf.xml b/framework/src/main/resources/MoquiDefaultConf.xml index d2ad0edee..cb515655f 100644 --- a/framework/src/main/resources/MoquiDefaultConf.xml +++ b/framework/src/main/resources/MoquiDefaultConf.xml @@ -49,14 +49,16 @@ - - - + + + + + @@ -67,7 +69,7 @@ - + @@ -75,13 +77,15 @@ - + - + diff --git a/framework/src/main/resources/log4j2.xml b/framework/src/main/resources/log4j2.xml index 78692f3bc..c31011e18 100644 --- a/framework/src/main/resources/log4j2.xml +++ b/framework/src/main/resources/log4j2.xml @@ -35,15 +35,15 @@ https://logging.apache.org/log4j/2.x/manual/layouts.html#PatternLayout + + + - - - diff --git a/framework/src/start/java/MoquiStart.java b/framework/src/start/java/MoquiStart.java index 031366779..73010a320 100644 --- a/framework/src/start/java/MoquiStart.java +++ b/framework/src/start/java/MoquiStart.java @@ -282,6 +282,20 @@ public static void main(String[] args) throws IOException { // NOTE DEJ20210520: now always using StartClassLoader because of breaking classloader changes in 9.4.37 (likely from https://github.com/eclipse/jetty.project/pull/5894) webappClass.getMethod("setClassLoader", ClassLoader.class).invoke(webapp, moquiStartLoader); + // handle webapp_session_cookie_max_age with setInitParameter (1209600 seconds is about 2 weeks 60 * 60 * 24 * 14) + String sessionMaxAge = System.getenv("webapp_session_cookie_max_age"); + if (sessionMaxAge != null && !sessionMaxAge.isEmpty()) { + Integer maxAgeInt = null; + try { maxAgeInt = Integer.parseInt(sessionMaxAge); } + catch (Exception e) { System.out.println("Found webapp_session_cookie_max_age env var with invalid number, ignoring: " + sessionMaxAge); } + + if (maxAgeInt != null) { + System.out.println("Setting Servlet Session Max Age based on webapp_session_cookie_max_age " + maxAgeInt); + webappClass.getMethod("setInitParameter", String.class, String.class) + .invoke(webapp, "org.eclipse.jetty.servlet.MaxAge", maxAgeInt.toString()); + } + } + // WebSocket Object wsContainer = wsInitializerClass.getMethod("configure", scHandlerClass, wsInitializerConfiguratorClass).invoke(null, webapp, null); webappClass.getMethod("setAttribute", String.class, Object.class).invoke(webapp, "javax.websocket.server.ServerContainer", wsContainer); diff --git a/framework/src/test/groovy/EntityNoSqlCrud.groovy b/framework/src/test/groovy/EntityNoSqlCrud.groovy index 667e5db8d..262af5ec4 100644 --- a/framework/src/test/groovy/EntityNoSqlCrud.groovy +++ b/framework/src/test/groovy/EntityNoSqlCrud.groovy @@ -121,12 +121,11 @@ class EntityNoSqlCrud extends Specification { def "ELI find TestNoSqlEntity"() { when: - EntityListIterator eli EntityList partialEl = null EntityValue first = null - try { - eli = ec.entity.find("moqui.test.TestNoSqlEntity") - .orderBy("-testNumberInteger").iterator() + try (EntityListIterator eli = ec.entity.find("moqui.test.TestNoSqlEntity") + .orderBy("-testNumberInteger").iterator()) { + partialEl = eli.getPartialList(0, 100, false) @@ -134,10 +133,7 @@ class EntityNoSqlCrud extends Specification { first = eli.next() } catch (Exception e) { logger.error("partialEl error", e) - } finally { - if (eli != null) eli.close() } - // logger.warn("partialEl.size() ${partialEl.size()} first value ${first}") then: diff --git a/framework/src/test/groovy/L10nFacadeTests.groovy b/framework/src/test/groovy/L10nFacadeTests.groovy index fbe4feef0..746015e09 100644 --- a/framework/src/test/groovy/L10nFacadeTests.groovy +++ b/framework/src/test/groovy/L10nFacadeTests.groovy @@ -98,7 +98,7 @@ class L10nFacadeTests extends Specification { ec.l10n.formatCurrency(new BigDecimal("12.34"), "USD", 2) == '$12.34' ec.l10n.formatCurrency(new BigDecimal("43.21"), "GBP", 2) in ["GBP43.21", "£43.21"] ec.user.setLocale(Locale.UK) - ec.l10n.formatCurrency(new BigDecimal("12.34"), "USD", 2) in ["USD12.34", 'US$12.34'] + ec.l10n.formatCurrency(new BigDecimal("12.34"), "USD", 2) in ["USD12.34", '$12.34'] ec.l10n.formatCurrency(new BigDecimal("43.21"), "GBP", 2) == "\u00A343.21" cleanup: diff --git a/framework/template/XmlActions.groovy.ftl b/framework/template/XmlActions.groovy.ftl index 66a235934..39236c122 100644 --- a/framework/template/XmlActions.groovy.ftl +++ b/framework/template/XmlActions.groovy.ftl @@ -15,6 +15,8 @@ import static org.moqui.util.ObjectUtilities.* import static org.moqui.util.CollectionUtilities.* import static org.moqui.util.StringUtilities.* import java.sql.Timestamp +import java.sql.Time +import java.time.* // these are in the context by default: ExecutionContext ec, Map context, Map result <#visit xmlActionsRoot/> diff --git a/framework/xsd/common-types-3.xsd b/framework/xsd/common-types-3.xsd index f6932bf3f..b92f3b03d 100644 --- a/framework/xsd/common-types-3.xsd +++ b/framework/xsd/common-types-3.xsd @@ -103,6 +103,9 @@ along with this software (see the LICENSE.md file). If not, see + + + diff --git a/framework/xsd/moqui-conf-3.xsd b/framework/xsd/moqui-conf-3.xsd index fa88f3b45..d060a6176 100644 --- a/framework/xsd/moqui-conf-3.xsd +++ b/framework/xsd/moqui-conf-3.xsd @@ -49,6 +49,7 @@ along with this software (see the LICENSE.md file). If not, see + @@ -60,6 +61,14 @@ along with this software (see the LICENSE.md file). If not, see table for moqui.basic.Enumeration). Empty or 'none' means load nothing, use 'all' to load all found data files regardless of type. + + Comma-separated list of data file types to load on start. Empty or 'none' means load nothing. + Does not run if empty-db-load runs. + + + Comma-separated list of component names to load on start, used with on-start-load-types. + Does not run if empty-db-load runs. + The maximum size of the worker queue. diff --git a/framework/xsd/xml-form-3.xsd b/framework/xsd/xml-form-3.xsd index eb3f22562..c2fe0937b 100644 --- a/framework/xsd/xml-form-3.xsd +++ b/framework/xsd/xml-form-3.xsd @@ -897,6 +897,10 @@ along with this software (see the LICENSE.md file). If not, see Specifies the currency uomId (ISO code) used to format the value. Will only format as currency if this is specified. + + When currency-unit-field has value, defines whether currency symbol will be + hidden or displayed normally. + Used to format the output of Number/Time/Date/Timestamp/etc objects. With auto-fields-service will inherit from service parameter. @@ -1088,6 +1092,10 @@ along with this software (see the LICENSE.md file). If not, see If applicable for the editor-type a screenThemeId for the CSS (resourceTypeEnumId=STRT_STYLESHEET) files to use for the editor context, defaults to active screen theme. + + Auto-grow the text area to fit the contents; + currently only supported in qvt mode (Vue + Quasar) +