Skip to content

Function I found useful - blockwise indexed task submission #35

@UH-Comet-Group

Description

@UH-Comet-Group

I found this function useful - sort of a blockwise pdotimes. One could do the same with map, but
this turned out to be simpler to use for me. If I'm not missing something, it might be a useful standard functionality (after changing the awful name).

I think it's not quite the same as pdotimes, because pdotimes calls its tasks by singles,
so that any overheads are repeated. Below, the task given by func can have expensive overheads.

;; utility function that takes a task indexed i1..i2, and
;; calls (func j1 j2) in parallel to span ranges of i1,i2

(defun chop-up-indexed-tasks-into-threads (i1 i2 func &key (nthreads nil))
  (loop with ncpu = (or nthreads (lparallel:kernel-worker-count))
	with ntot = (- i2 i1)
	with nblock = (ceiling ntot ncpu) ;; number in a block
	with ithread = 0
	with channel = (lparallel:make-channel)
	for j1 = i1 then (+ 1 j1 nblock)
	for j2 = (min (+ j1 nblock) i2)
	do (lparallel:submit-task channel func j1 j2)
	   (incf ithread)
	until (= j2 i2)
	finally
	   (loop for k below ithread
		 do (lparallel:receive-result channel))))

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions