In parallel computing a computational task is typically broken down into many sub-tasks that can be processed independently and whose results are combined upon completion. It is also called multiprocessing. In serial computing a single core multitasks all or part of a series of instructions for a short period of time before switching to another task, giving the appearance of all the tasks running simultaneously even though they run one at a time.
Parallel computing and processing involves more than one computational core (CPU or GPU, even FPGA or ASIC) where one task is broken up into sub-tasks or multiple different tasks are executed in parallel in a shorter amount of time than would be required to execute on a single core, assuming a well written program.
Each sub-task or entirely different task must be capable of being executed for some period of time without interaction with the other parallel running tasks, otherwise a deadlock would occur where one running task must wait for the result of another.
Concurrent computing differs from parallel computing in that with concurrent computing each task can run without reliance on another, if one input (from one user, for example) isn't available the routine can continue to solicit other input. Once the input has been processed it can be sent to it's output destination.
Parallel computing, on the other hand, requires that the tasks that were broken up (or the entirely separate tasks, whose output is to be combined) be synchronized to end in-order, or await other tasks which must be completed first. When complex tasks can be divided between more cores and simpler tasks ran on fewer cores, with the completion times synchronized, the execution time can be reduced over running each instruction serially on one core (there being a relatively low upper limit on the clock speed of each core, but an extremely high practical limit on the total number of cores).
