Rendering Images with Renderer

[lychenyoko]

Hi, I have been trying to understand the effectiveness of imitative learning by using the face renderer Face3D() to render some faces.

Here are my codes:

from renderer.face_decoder import Face3D
from training.training_utils import z_to_lambda_mapping

FaceRender = Face3D()

minibatch_in = tf.placeholder(tf.int32, name='minibatch_in', shape=[])
resolution = tf.placeholder(tf.float32, name='resolution', shape=[]) # Try a tf-placeholder

latents = tf.random_normal([minibatch_in,128+32+16+3])
INPUTcoeff = z_to_lambda_mapping(latents)
INPUTcoeff_w_t = tf.concat([INPUTcoeff,tf.zeros([minibatch_in,3])], axis = 1)
render_img,render_mask,render_landmark,_ = FaceRender.Reconstruction_Block(INPUTcoeff_w_t,resolution,minibatch_in,progressive=True)

sess = tf.Session()
init = tf.global_variables_initializer()
sess.run(init)

res = 32
minibatch = 4
r_img, r_mask_, r_lmark = sess.run([render_img, render_mask, render_landmark], 
                                    {resolution: res, minibatch_in: minibatch})

I suppose r_img, r_mask, r_lmark would be the rendered the stuffs, yet it returns an error:

---------------------------------------------------------------------------
InvalidArgumentError                      Traceback (most recent call last)
/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tensorflow/python/client/session.py in _do_call(self, fn, *args)
   1333     try:
-> 1334       return fn(*args)
   1335     except errors.OpError as e:

/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tensorflow/python/client/session.py in _run_fn(feed_dict, fetch_list, target_list, options, run_metadata)
   1318       return self._call_tf_sessionrun(
-> 1319           options, feed_dict, fetch_list, target_list, run_metadata)
   1320 

/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tensorflow/python/client/session.py in _call_tf_sessionrun(self, options, feed_dict, fetch_list, target_list, run_metadata)
   1406         self._session, options, feed_dict, fetch_list, target_list,
-> 1407         run_metadata)
   1408 

InvalidArgumentError: Input to reshape is a tensor with 428508 values, but the requested shape has 1714032
	 [[{{node cond/cond/cond/Reshape}} = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](cond/cond/cond/cond/Reshape/Switch:1, cond/cond/cond/Reshape_1/shape)]]
	 [[{{node cond/cond/cond/cond/cond/GatherV2_2/_111}} = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_3747_cond/cond/cond/cond/cond/GatherV2_2", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

During handling of the above exception, another exception occurred:

InvalidArgumentError                      Traceback (most recent call last)
<ipython-input-9-63fad876710d> in <module>
      2 minibatch = 4
      3 r_img, r_mask_, r_lmark = sess.run([render_img, render_mask, render_landmark], 
----> 4                                     {resolution: res, minibatch_in: minibatch})

/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tensorflow/python/client/session.py in run(self, fetches, feed_dict, options, run_metadata)
    927     try:
    928       result = self._run(None, fetches, feed_dict, options_ptr,
--> 929                          run_metadata_ptr)
    930       if run_metadata:
    931         proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)

/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tensorflow/python/client/session.py in _run(self, handle, fetches, feed_dict, options, run_metadata)
   1150     if final_fetches or final_targets or (handle and feed_dict_tensor):
   1151       results = self._do_run(handle, final_targets, final_fetches,
-> 1152                              feed_dict_tensor, options, run_metadata)
   1153     else:
   1154       results = []

/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tensorflow/python/client/session.py in _do_run(self, handle, target_list, fetch_list, feed_dict, options, run_metadata)
   1326     if handle is None:
   1327       return self._do_call(_run_fn, feeds, fetches, targets, options,
-> 1328                            run_metadata)
   1329     else:
   1330       return self._do_call(_prun_fn, handle, feeds, fetches)

/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tensorflow/python/client/session.py in _do_call(self, fn, *args)
   1346           pass
   1347       message = error_interpolation.interpolate(message, self._graph)
-> 1348       raise type(e)(node_def, op, message)
   1349 
   1350   def _extend_graph(self):

InvalidArgumentError: Input to reshape is a tensor with 428508 values, but the requested shape has 1714032
	 [[node cond/cond/cond/Reshape (defined at ../DiscoFaceGAN/renderer/face_decoder.py:303)  = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](cond/cond/cond/cond/Reshape/Switch:1, cond/cond/cond/Reshape_1/shape)]]
	 [[{{node cond/cond/cond/cond/cond/GatherV2_2/_111}} = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_3747_cond/cond/cond/cond/cond/GatherV2_2", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

Caused by op 'cond/cond/cond/Reshape', defined at:
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/ipykernel_launcher.py", line 16, in <module>
    app.launch_new_instance()
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/traitlets/config/application.py", line 664, in launch_instance
    app.start()
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/ipykernel/kernelapp.py", line 619, in start
    self.io_loop.start()
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tornado/platform/asyncio.py", line 199, in start
    self.asyncio_loop.run_forever()
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/asyncio/base_events.py", line 442, in run_forever
    self._run_once()
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/asyncio/base_events.py", line 1462, in _run_once
    handle._run()
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/asyncio/events.py", line 145, in _run
    self._callback(*self._args)
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tornado/ioloop.py", line 688, in <lambda>
    lambda f: self._run_callback(functools.partial(callback, future))
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tornado/ioloop.py", line 741, in _run_callback
    ret = callback()
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tornado/gen.py", line 814, in inner
    self.ctx_run(self.run)
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tornado/gen.py", line 162, in _fake_ctx_run
    return f(*args, **kw)
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tornado/gen.py", line 775, in run
    yielded = self.gen.send(value)
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 358, in process_one
    yield gen.maybe_future(dispatch(*args))
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tornado/gen.py", line 234, in wrapper
    yielded = ctx_run(next, result)
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tornado/gen.py", line 162, in _fake_ctx_run
    return f(*args, **kw)
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 261, in dispatch_shell
    yield gen.maybe_future(handler(stream, idents, msg))
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tornado/gen.py", line 234, in wrapper
    yielded = ctx_run(next, result)
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tornado/gen.py", line 162, in _fake_ctx_run
    return f(*args, **kw)
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 538, in execute_request
    user_expressions, allow_stdin,
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tornado/gen.py", line 234, in wrapper
    yielded = ctx_run(next, result)
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tornado/gen.py", line 162, in _fake_ctx_run
    return f(*args, **kw)
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/ipykernel/ipkernel.py", line 302, in do_execute
    res = shell.run_cell(code, store_history=store_history, silent=silent)
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/ipykernel/zmqshell.py", line 539, in run_cell
    return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2867, in run_cell
    raw_cell, store_history, silent, shell_futures)
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2895, in _run_cell
    return runner(coro)
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/IPython/core/async_helpers.py", line 68, in _pseudo_sync_runner
    coro.send(None)
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 3072, in run_cell_async
    interactivity=interactivity, compiler=compiler, result=result)
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 3263, in run_ast_nodes
    if (await self.run_code(code, result,  async_=asy)):
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 3343, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-7-d5186183c6b1>", line 4, in <module>
    render_img,render_mask,render_landmark,_ = FaceRender.Reconstruction_Block(INPUTcoeff_w_t,resolution,minibatch_in,progressive=True)
  File "../DiscoFaceGAN/renderer/face_decoder.py", line 65, in Reconstruction_Block
    lambda:
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func
    return func(*args, **kwargs)
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2097, in cond
    orig_res_f, res_f = context_f.BuildCondBranch(false_fn)
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 1930, in BuildCondBranch
    original_result = fn()
  File "../DiscoFaceGAN/renderer/face_decoder.py", line 67, in <lambda>
    lambda:
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func
    return func(*args, **kwargs)
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2097, in cond
    orig_res_f, res_f = context_f.BuildCondBranch(false_fn)
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 1930, in BuildCondBranch
    original_result = fn()
  File "../DiscoFaceGAN/renderer/face_decoder.py", line 69, in <lambda>
    lambda:
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func
    return func(*args, **kwargs)
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2086, in cond
    orig_res_t, res_t = context_t.BuildCondBranch(true_fn)
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 1930, in BuildCondBranch
    original_result = fn()
  File "../DiscoFaceGAN/renderer/face_decoder.py", line 68, in <lambda>
    tf.cond(res<=32, lambda:self.Render_block(face_shape_t,norm_r,face_color,self.facemodel,32,16),
  File "../DiscoFaceGAN/renderer/face_decoder.py", line 303, in Render_block
    face_shape = tf.reshape(face_shape,[batchsize,n_vex,3])
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 6482, in reshape
    "Reshape", tensor=tensor, shape=shape, name=name)
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func
    return func(*args, **kwargs)
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3274, in create_op
    op_def=op_def)
  File "/opt/conda/envs/discofacegan_env/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1770, in __init__
    self._traceback = tf_stack.extract_stack()

InvalidArgumentError (see above for traceback): Input to reshape is a tensor with 428508 values, but the requested shape has 1714032
	 [[node cond/cond/cond/Reshape (defined at ../DiscoFaceGAN/renderer/face_decoder.py:303)  = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](cond/cond/cond/cond/Reshape/Switch:1, cond/cond/cond/Reshape_1/shape)]]
	 [[{{node cond/cond/cond/cond/cond/GatherV2_2/_111}} = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_3747_cond/cond/cond/cond/cond/GatherV2_2", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

Could you help me with this? Thanks in advance!

[YuDeng]

In FaceRender.Reconstruction_Block I hard code the batchsize for different image resolution:

DiscoFaceGAN/renderer/face_decoder.py

Line 59 in 0530bd1

if progressive:

The reason for this hard coding is that tf_mesh_render (the version I used) receives data with fixed batchsize in the rendering process.

In your test case, the image resolution is 32 and the batchsize is set to 4, which differs from the batchsize for resolution 32x32 thus causing the error.